Categories
Life Skills Technology Tutorial Ubuntu Ubuntu Touch

RECOVERING DATA FROM YOUR ANDROID DEVICE USING FOREMOST AND UBUNTU

“Ooops. I deleted my photos”

Well, I thought this would be a bit easier than it was but such is life – HARD…May this tutorial save many or all of you lots of time. Probably you can do all this with Kali, Debian, or other distributions that have the same commands, but I’m ubuntu so ubuntu it will be.

My goal here was pretty simple. My sister wiped her photos from her phone (or someone she knows…) and they are gone. One would think they would save somewhere in google’s creepy backup network, but seems not so. Perhaps when you press ‘delete’ on your Android device they delete first from your device then from the google servers as they sync? That’s actually good thing for security reasons and privacy – which is why I doubt it works like that (ha). Anyway, I use Ubuntu Touch so I don’t know. But I do know that Nextcloud is awesome and I believe you can set it up for manual backups so it doesn’t happen automatically and you can probably set it up just fine on Android, too. This would be ideal setup so if you accidentally delete stuff like this the deletion won’t make it to the server and you could backup from that…. Anyway, my sister wanted these photos and I like learning so I started to review how to do it from a tutorial I never finished back in 2013. At that time I had wiped all my emails in Thunderbird, and I recall I did successfully get them all back.

The first interesting thing I encountered is that when I plugged the android device in my ubuntu computer – unlike my Ubuntu Touch mobile device – an Android device ‘mounts’ but doesn’t show up as a usual sdb, sdc, sdd, etc drive in File Manager system. The reason for this is that Android apparently uses ‘MTP’ and this was the source of my initial pain. If you are interested, here is a link about my mtp journey, only because I feel I don’t want to waste the time I spent logging it all and perhaps one day it will be useful related info.

Cloning the Drive with ddrescue!

To clone the SD card of the device we are going to use a tool called ddrescue, which annoyingly is called ‘gddrescue’ in the Ubuntu packages, yet, when you run it is ‘ddrescue’, not ‘gddrescue’. I wasted a solid 15 minutes trying to figure that one out…

  1. Install dd rescue on ubuntu: sudo apt install gddrescue
  2. learn how it works and to make sure you have the right tool: man ddrescue (again, not ‘man gddrescue’)(man!)

This blog post was good except that lshw -C disk only shows discs and direct USBs I guess but not SD cards, and since my sd card is in the sd card adaptor directly in my laptop, I’m going to use lsblk which will show the SD cards and their ‘mount points’. If you haven’t seen these before, they typically show up as an entry starting with ‘mmc’.

  1. Identify your SD card mount logical whatever: lsblk
    Mine is ‘mmcblk0’ but yours could be different. If there is a ‘tree’ of partitions, just choose the top level one for the card.

Documentation shows the basic command systax for ddrescue as: ddrescue [options] infile outfile [logfile]

They also had this example:

root# ddrescue -f -n /dev/[baddrive] /root/[imagefilename].img /root/recovery.log

I will change the above example to the following, but you’ll adjust yours accordingly:
sudo ddrescue /dev/mmcblk0 ~/samsungs7.img

What I’m hoping this will do is burn a copy of the sd card to an image file which I will then be able to carve. If you do your research you’ll find that it is always recommended to clone a drive and do forensics work on it rather than do the work directly on the ‘active’ drive. I’ve removed options because I want the most thorough image creation. Disclaimer: I have not studied ddrescue in detail, so always do your own deeper dive if you have time.

Note 1 before you run the command to clone: I quickly found out at this point that my laptop didn’t have enough memory so decided to clone the drive straight to another USB pendrive of same size to keep my laptop free from having a bad situation of running out of memory. However, at the same time I thought I would take my advice and use a drive of the same size of the cloned image. Wrong move. Even though advice online says that you need ‘a drive of the same size or larger’, there must be tiny differences of size between the manufacturers because just as ddrescue was finishing the job of cloning, it stopped and told me the usb drive ran out of memory. I ended up pointing it to my big storage drive. So, my advice is this: output your cloned image to a drive LARGER than the size of the drive that is being cloned. This could save you time, headaches, and confusion.

Note 2 before you begin to clone: be careful that your path for the output image file is a ‘real storage path’ on your machine, and do not accidentally include something such as ‘/dev/sdc’ as you will get the ‘this is not a directory’ warning. If you get this warning simply check the path and adjust accordingly.

The following is my more layman’s example of the command I used successfully showing in SD card input device with the Android data on it (“mmcblk0”), the output image file going to my big storage drive (“HardDriveName”) into a directory on that drive that I have already created (For example a directory called ‘samsung’), and then finally creating a clone of the drive called in that directory (“cloneImageName.img”):
sudo ddrescue /dev/mmcblk0 /media/user/HardDriveName/path/to/place/to/store/clone/cloneImageName.img

Update the above according to your paths, of course, including the mmcblk0 as yours might be different – lsblk to confirm…

When the ddrescue command is running it looked like this for me:

GNU ddrescue 1.22
ipos: 67698 kB, non-trimmed: 0 B, current rate: 43384 kB/s
opos: 67698 kB, non-scraped: 0 B, average rate: 22566 kB/s
non-tried: 15957 MB, bad-sector: 0 B, error rate: 0 B/s
rescued: 67698 kB, bad areas: 0, run time: 2s
pct rescued: 0.42%, read errors: 0, remaining time: 11m
time since last successful read: n/a
Copying non-tried blocks… Pass 1

Carving the files with Foremost!

Good job. We now have the clone .img file and now comes the hopeful recovery stuff. We now will start rippping through the clone to see what we can find.

  1. Get software called foremost: sudo apt install foremost
  2. Learn about it: man foremost

In this case I’ll be trying to carve out picture files. It’s a Samsung and I happen to know all photos are .jpg so i’m going to narrow down this carving to .jpg only, to speed it up.

foremost -i /media/user/HardDriveName/path/to/place/to/store/clone/cloneImageName.img -o /media/user/HardDriveName/path/to/place/to/save/restored/pictures -t jpg -v

You can seen now my input file / target file is the same one we created in the cloning section above:
/media/user/HardDriveName/path/to/place/to/store/clone/cloneImageName.img

The place where we’ll save the carved files is:
/media/user/HardDriveName/path/to/place/to/save/restored/pictures

I stuck -v on the end to make it more obvious (verbose) in the terminal what’s happening. I recommend the same.

When everything is complete, by the way, Foremost creates and saves a file in the carved files directory called ‘audit.txt’. Then, a specific directory is createed called ‘jpg’ which will then house the files. This is nice and I didn’t know this happens before I began so I wasted some time creating directories with custom file-type names. No need. Foremost does this for you.

The process may take a long while so pack a lunch…

Conclusion

These two tools ddrescue (for cloning) and Foremost (for carving) are great and not so hard to use. Foremost successfully found some deleted photos, however, it was unable to find deleted videos. In fact, Foremost didn’t find a single video on the SD card which makes me wonder if it worked or if I did something wrong. I specified the .mp4 extension but nothing. I will continue to look into this, but for now, if you need to recover photos, I can tell you this worked great for me.

In fact, I ended up studying and using a second piece of carving software called scalpel which turned out also to be great, but it seems a little less simple to use. I created an similar instructional blog about scalpel file carving on ubuntu if you’d like to try that one too.

Categories
Technology Ubuntu Ubuntu Touch

Log notes from my MTP android ubuntu learning day

Before you read this. Big disclaimer. This post is written in super crappy style and is nothing more than my personal notes as I tried to figure out MTP. The only reason I’m even putting this post online is because a few of the things I learned below seem like they will be relevant for future ubuntu touch / android porting things. Even skimming over the process I went through seems useful. However, if you are looking for a good tutorial about how to actually do something with MTP / android / ubuntu, this is not that. Hopefully it will provide some value to someone though.

With that out of the way, here comes the crap:

I went down a long road installing ‘MTP://’ mount thing with sudo apt install mtp-tools (very few tutorials on this one) but after it was all said and done, I don’t think it was required because it seemed that all the entries for the ‘udev/rules.d’ files were already complete.

I had assumed you need to plug in the device and clone the internal hard drive (EMMC) via USB, but after some chats with some software communities it seems that ‘it doesn’t work like that’ and the situation is as follows (feel free to comment below otherwise to help me and other readers):

  • Everything nowadays is related to the SD card
  • You have to clone the SD card
  • You have to do forensics / file carving on the cloned drive image

I still feel there has to be a way to carve stuff from the EMMC via MTP, but that perhaps is for another tutorial blog. If you want to look into this, I will provide my learning here, but if you have an Android device with the userdata on the SD card (this is probably the case), then skip on to the following section:

  • How to ‘mount’ an Android device? With MTP
  • What is MTP? It’s this
  • If you are on ubuntu and try lsusb and can see something like this, you are probably already set up well with MTP:
    “Samsung Electronics Co., Ltd Galaxy (MTP)”

Un-mounting android.
un-plugging android.
lsusb
good. I now get “Samsung Electronics Co., Ltd Galaxy (MTP)”

Now to figure out how to find that drive… how does one mount via mtp-tools?
It seems like most stuff is from circa 2014 which is possibly old and scary at this point.
man mtp-tools

nice. tools… manly tools.

wow. bad tool page. looks like MTP tools don’t really ‘mount’ the drive as a drive but just help you communicate with the drive. Not what we’re trying to do here. Jumping up a level.

This guy seems to have figured it out in this video I watched it, felt scared, then read the following code in his notes and this seemed to make the fear go away. Now, we already have mtp-tools installed from above so I’m going to assume that we don’t need to do all of it however, commments on YT from a week ago show ‘thanks man, it worked’ so… yeah. Maybe I’ll just follow it to a tee:

sudo apt-get install libmtp-common mtp-tools libmtp-dev libmtp-runtime libmtp9

sudo apt-get dist-upgrade

sudo nano /etc/fuse.conf

lsusb

sudo nano /lib/udev/rules.d/69-mtp.rules

# Device
ATTR{idVendor}=="XXXX", ATTR{idProduct}=="YYYY", SYMLINK+="libmtp-%k", ENV{ID_MTP_DEVICE}="1", ENV{ID_MEDIA_PLAYER}="1"

sudo nano /etc/udev/rules.d/51-android.rules

ATTR{idVendor}==”XXXX”, ATTR{idProduct}==”YYYY”, MODE=”0666″

sudo service udev restart

sudo reboot

I’m going to walk through these commands one at a time and freshen them up if they need freshining.

sudo apt install libmtp-common <– not needed. shows current
sudo apt install libmtp-dev <– yes, required, not currently installed
sudo apt install libmtp-runtime <– no, not required, showing current
sudo apt install libmtp9 <– no, not required, shows current.

Therefore, the first line could be:

sudo apt install libmtp-dev

I’m a little concerned about running sudo apt dist-upgrade because I think that upgrades your entire current distro from, say 18.04 to 18.10… that seems a bit intense so I’m going to try to skip that one. Instead I did:

sudo apt update and sudo apt upgrade

Since we already have mtp-tools installed and we know that lsusb is showing the right stuff “Bus 002 Device 036: ID 04e8:6860 Samsung Electronics Co., Ltd Galaxy (MTP)” i think we can move on to adding the rules

blast. it looks like this one rule location might be out of date sudo nano /etc/udev/rules.d/51-android.rules

cd /etc/udev/rules.d/ to search for it…

not there… maybe needs reboot. Rebooting….

sudo nano /etc/fuse.conf
remove comment (#) ‘user_allow_other’
control x to save changes

sudo nano /lib/udev/rules.d/69-libmtp.rules <– note, this is different from tutorial, couldn’t find his file name and assumed this is updated name
1m15s video is useful to watch as he pastes

connect phone
do an lsusb in one terminal
open new terminal window so you can see both together

take this, and update the XXXX to your device things as listed in lsusb in the other window

# Device
ATTR{idVendor}=="XXXX", ATTR{idProduct}=="YYYY", SYMLINK+="libmtp-%k", ENV{ID_MTP_DEVICE}="1", ENV{ID_MEDIA_PLAYER}="1"

mine will become, for example AS DEMONSTRACTED at 1m44s in video with blue and red arrows (great visual, by the way!)
idVendor is the first 4 characters before the colon
idProduct are the second 4 characters after the colon.

# SAMSUNG GALAXY 7
ATTR{idVendor}=="04e8", ATTR{idProduct}=="6860", SYMLINK+="libmtp-%k", ENV{ID_MTP_DEVICE}="1", ENV{ID_MEDIA_PLAYER}="1"

I like to do stuff in text editors and make sure it’s good before pasting into terminals but up to you.

Oh CRAZY. it waslready in this sudo nano /lib/udev/rules.d/69-libmtp.rules

now what… hmm

well I guess we can say that was ‘step 1 to make sure that your device is recognized by MTP’. Now that is good we must do next stuff

sudo nano /etc/udev/rules.d/51-android.rules

not there. but foudn it here:

sudo /lib/udev/rules.d

so:

`/lib/udev/rules.d/51-

well that was all a waste of time, kind of. Turns out that the SD card is the mount point of the EMMC memory anyways (apparently) so I discovered there was an SD card in device and may work….