Categories
Technology Tutorial Ubuntu Ubuntu Touch

RECOVERING FILES USING SCALPEL AND UBUNTU

“Ooops. I deleted my photos”

I used foremost in this blog to try to recover deleted mp4 video files (for my sister) using ubuntu and the software called Foremost, but it wouldn’t find the video files. It found a bunch of photos but not her supposed videos.

Then, I thought maybe the tool wasn’t capable so I tried to figure out: Scalpel which is ‘the other’ major tool besides foremost.

But how to use it?

This video is an absolutely great way to get started doing the basics. Definitely take the time to watch it as it will give you a good birds eye view not just about carving an image file but also how to carve a device (ie. hard drive, usb drive) connected to the machine.

Also, the man scalpel was pretty useful, however, I still couldn’t figure out

a) where the ever-important scalpel.conf file was and
b) how to do mp4 which was (oddly) missing from scalpel.conf

It was pretty surprising that mp4 was missing, that’s for sure…

However, I did find this blog which got me most of the way there in terms of basic learning. Interestingly the author did not make available the following code containing the mp4 header/footer stuff to simply copy and paste. However, I was able to manually type from the screenshot so that myself (and the world) can just copy and paste it. Here it is:

EDIT 200816 – See below the disclaimer before doing this. I was wrong..

mp4 y 30000000:70000000 \x46\x4c\x56\x01\x05\x00\x00\x00\x09\x00\x00\x00\x00\x12\x00\x02\x36

Important Disclaimer: I have no idea if this thing is right, or works, but the program itself, I can verify, ran perfectly and carved for files. No videos were ultimately carved in my situation, but I don’t think that is the fault of this software nor my copy/paste stuff above. It’s because I don’t think there actually were any videos on my drive to start with. But I wanted to be clear that I simply typed out another bloggers stuff and stuck it here.

I was wrong, it seems. I kept getting nothing at all which I thought was weird. Then I just decided to dump the hex stuff above into a hex-to-text editor thing and discovered it translates to “FLV”.. So then what I did was use the same hex editor and wrote ‘mp4’ in there, and got this output:

6d 70 34

Then I updated the scalpel.conf file to look like this:

mp4 y 30000000:70000000 6d 70 34

This started working so I appear on the right path. However files still not playable and came in about 100 small chunks per directory. Need correct scalpel.conf entry to carve an mp4 file… any help appreciated in comments below!

Then I finally found another blog which showed that the scalpel.conf file is located at /etc/scalpel. Now you don’t have to spend an hour to learn that!

The first thing I did once I had what I needed to paste, and knew where to paste it, was edit the scalpel.conf file with the following command, and then copy/paste the above ‘stuff’ at the very bottom of the file and then ctrl+x, etc, to save changes in the file:

sudo nano /etc/scalpel/scalpel.conf

Once that is done, we need to run the command. Even this kind of messed me up with a few of the tutorials and even the man pages I read. Here is the syntax of what you’ll need to start it running with my breakdown of what each component is following:

sudo scalpel -c /etc/scalpel/scalpel.conf -o /path/to/output/diretory/directoryname /path/to/location/of/image.img

Quick explanation of this so you can plug in your details into the command to get carving:

sudo – it requires super user permissions (and your password)
-c – this tag points to the scalpel config file you edited above. If you want, you can move it, but if you didn’t move it this path in my example above is correct
-o – this points to your output path where ou want the recovered files to go
directoryname – at the end of your path add a useful name you’ll remember, as this directory will be created and the recovered files placed inside if it works
/path/to/location/of/image.img – likely you have already cloned the drive and are not using the original (recommended for many reasons) so this is the path to where on your machine this image file is located.
image.img – represents the name of your image, whatever it is.

So once you customize your details and hit ‘go’ you should be able to start carving files. Also, if you want to carve for more than just mp4s you can un-comment the options in the scalpel.conf file, or add others and they will all run at the same time once you start.

I found that Foremost found more .jpg than Scalpel but I might have configured one or both of them incorrectly…

Hope this helps you recover some lost goods!

Categories
Life Skills Technology Tutorial Ubuntu Ubuntu Touch

RECOVERING DATA FROM YOUR ANDROID DEVICE USING FOREMOST AND UBUNTU

“Ooops. I deleted my photos”

Well, I thought this would be a bit easier than it was but such is life – HARD…May this tutorial save many or all of you lots of time. Probably you can do all this with Kali, Debian, or other distributions that have the same commands, but I’m ubuntu so ubuntu it will be.

My goal here was pretty simple. My sister wiped her photos from her phone (or someone she knows…) and they are gone. One would think they would save somewhere in google’s creepy backup network, but seems not so. Perhaps when you press ‘delete’ on your Android device they delete first from your device then from the google servers as they sync? That’s actually good thing for security reasons and privacy – which is why I doubt it works like that (ha). Anyway, I use Ubuntu Touch so I don’t know. But I do know that Nextcloud is awesome and I believe you can set it up for manual backups so it doesn’t happen automatically and you can probably set it up just fine on Android, too. This would be ideal setup so if you accidentally delete stuff like this the deletion won’t make it to the server and you could backup from that…. Anyway, my sister wanted these photos and I like learning so I started to review how to do it from a tutorial I never finished back in 2013. At that time I had wiped all my emails in Thunderbird, and I recall I did successfully get them all back.

The first interesting thing I encountered is that when I plugged the android device in my ubuntu computer – unlike my Ubuntu Touch mobile device – an Android device ‘mounts’ but doesn’t show up as a usual sdb, sdc, sdd, etc drive in File Manager system. The reason for this is that Android apparently uses ‘MTP’ and this was the source of my initial pain. If you are interested, here is a link about my mtp journey, only because I feel I don’t want to waste the time I spent logging it all and perhaps one day it will be useful related info.

Cloning the Drive with ddrescue!

To clone the SD card of the device we are going to use a tool called ddrescue, which annoyingly is called ‘gddrescue’ in the Ubuntu packages, yet, when you run it is ‘ddrescue’, not ‘gddrescue’. I wasted a solid 15 minutes trying to figure that one out…

  1. Install dd rescue on ubuntu: sudo apt install gddrescue
  2. learn how it works and to make sure you have the right tool: man ddrescue (again, not ‘man gddrescue’)(man!)

This blog post was good except that lshw -C disk only shows discs and direct USBs I guess but not SD cards, and since my sd card is in the sd card adaptor directly in my laptop, I’m going to use lsblk which will show the SD cards and their ‘mount points’. If you haven’t seen these before, they typically show up as an entry starting with ‘mmc’.

  1. Identify your SD card mount logical whatever: lsblk
    Mine is ‘mmcblk0’ but yours could be different. If there is a ‘tree’ of partitions, just choose the top level one for the card.

Documentation shows the basic command systax for ddrescue as: ddrescue [options] infile outfile [logfile]

They also had this example:

root# ddrescue -f -n /dev/[baddrive] /root/[imagefilename].img /root/recovery.log

I will change the above example to the following, but you’ll adjust yours accordingly:
sudo ddrescue /dev/mmcblk0 ~/samsungs7.img

What I’m hoping this will do is burn a copy of the sd card to an image file which I will then be able to carve. If you do your research you’ll find that it is always recommended to clone a drive and do forensics work on it rather than do the work directly on the ‘active’ drive. I’ve removed options because I want the most thorough image creation. Disclaimer: I have not studied ddrescue in detail, so always do your own deeper dive if you have time.

Note 1 before you run the command to clone: I quickly found out at this point that my laptop didn’t have enough memory so decided to clone the drive straight to another USB pendrive of same size to keep my laptop free from having a bad situation of running out of memory. However, at the same time I thought I would take my advice and use a drive of the same size of the cloned image. Wrong move. Even though advice online says that you need ‘a drive of the same size or larger’, there must be tiny differences of size between the manufacturers because just as ddrescue was finishing the job of cloning, it stopped and told me the usb drive ran out of memory. I ended up pointing it to my big storage drive. So, my advice is this: output your cloned image to a drive LARGER than the size of the drive that is being cloned. This could save you time, headaches, and confusion.

Note 2 before you begin to clone: be careful that your path for the output image file is a ‘real storage path’ on your machine, and do not accidentally include something such as ‘/dev/sdc’ as you will get the ‘this is not a directory’ warning. If you get this warning simply check the path and adjust accordingly.

The following is my more layman’s example of the command I used successfully showing in SD card input device with the Android data on it (“mmcblk0”), the output image file going to my big storage drive (“HardDriveName”) into a directory on that drive that I have already created (For example a directory called ‘samsung’), and then finally creating a clone of the drive called in that directory (“cloneImageName.img”):
sudo ddrescue /dev/mmcblk0 /media/user/HardDriveName/path/to/place/to/store/clone/cloneImageName.img

Update the above according to your paths, of course, including the mmcblk0 as yours might be different – lsblk to confirm…

When the ddrescue command is running it looked like this for me:

GNU ddrescue 1.22
ipos: 67698 kB, non-trimmed: 0 B, current rate: 43384 kB/s
opos: 67698 kB, non-scraped: 0 B, average rate: 22566 kB/s
non-tried: 15957 MB, bad-sector: 0 B, error rate: 0 B/s
rescued: 67698 kB, bad areas: 0, run time: 2s
pct rescued: 0.42%, read errors: 0, remaining time: 11m
time since last successful read: n/a
Copying non-tried blocks… Pass 1

Carving the files with Foremost!

Good job. We now have the clone .img file and now comes the hopeful recovery stuff. We now will start rippping through the clone to see what we can find.

  1. Get software called foremost: sudo apt install foremost
  2. Learn about it: man foremost

In this case I’ll be trying to carve out picture files. It’s a Samsung and I happen to know all photos are .jpg so i’m going to narrow down this carving to .jpg only, to speed it up.

foremost -i /media/user/HardDriveName/path/to/place/to/store/clone/cloneImageName.img -o /media/user/HardDriveName/path/to/place/to/save/restored/pictures -t jpg -v

You can seen now my input file / target file is the same one we created in the cloning section above:
/media/user/HardDriveName/path/to/place/to/store/clone/cloneImageName.img

The place where we’ll save the carved files is:
/media/user/HardDriveName/path/to/place/to/save/restored/pictures

I stuck -v on the end to make it more obvious (verbose) in the terminal what’s happening. I recommend the same.

When everything is complete, by the way, Foremost creates and saves a file in the carved files directory called ‘audit.txt’. Then, a specific directory is createed called ‘jpg’ which will then house the files. This is nice and I didn’t know this happens before I began so I wasted some time creating directories with custom file-type names. No need. Foremost does this for you.

The process may take a long while so pack a lunch…

Conclusion

These two tools ddrescue (for cloning) and Foremost (for carving) are great and not so hard to use. Foremost successfully found some deleted photos, however, it was unable to find deleted videos. In fact, Foremost didn’t find a single video on the SD card which makes me wonder if it worked or if I did something wrong. I specified the .mp4 extension but nothing. I will continue to look into this, but for now, if you need to recover photos, I can tell you this worked great for me.

In fact, I ended up studying and using a second piece of carving software called scalpel which turned out also to be great, but it seems a little less simple to use. I created an similar instructional blog about scalpel file carving on ubuntu if you’d like to try that one too.

Categories
Technology Ubuntu Ubuntu Touch

Log notes from my MTP android ubuntu learning day

Before you read this. Big disclaimer. This post is written in super crappy style and is nothing more than my personal notes as I tried to figure out MTP. The only reason I’m even putting this post online is because a few of the things I learned below seem like they will be relevant for future ubuntu touch / android porting things. Even skimming over the process I went through seems useful. However, if you are looking for a good tutorial about how to actually do something with MTP / android / ubuntu, this is not that. Hopefully it will provide some value to someone though.

With that out of the way, here comes the crap:

I went down a long road installing ‘MTP://’ mount thing with sudo apt install mtp-tools (very few tutorials on this one) but after it was all said and done, I don’t think it was required because it seemed that all the entries for the ‘udev/rules.d’ files were already complete.

I had assumed you need to plug in the device and clone the internal hard drive (EMMC) via USB, but after some chats with some software communities it seems that ‘it doesn’t work like that’ and the situation is as follows (feel free to comment below otherwise to help me and other readers):

  • Everything nowadays is related to the SD card
  • You have to clone the SD card
  • You have to do forensics / file carving on the cloned drive image

I still feel there has to be a way to carve stuff from the EMMC via MTP, but that perhaps is for another tutorial blog. If you want to look into this, I will provide my learning here, but if you have an Android device with the userdata on the SD card (this is probably the case), then skip on to the following section:

  • How to ‘mount’ an Android device? With MTP
  • What is MTP? It’s this
  • If you are on ubuntu and try lsusb and can see something like this, you are probably already set up well with MTP:
    “Samsung Electronics Co., Ltd Galaxy (MTP)”

Un-mounting android.
un-plugging android.
lsusb
good. I now get “Samsung Electronics Co., Ltd Galaxy (MTP)”

Now to figure out how to find that drive… how does one mount via mtp-tools?
It seems like most stuff is from circa 2014 which is possibly old and scary at this point.
man mtp-tools

nice. tools… manly tools.

wow. bad tool page. looks like MTP tools don’t really ‘mount’ the drive as a drive but just help you communicate with the drive. Not what we’re trying to do here. Jumping up a level.

This guy seems to have figured it out in this video I watched it, felt scared, then read the following code in his notes and this seemed to make the fear go away. Now, we already have mtp-tools installed from above so I’m going to assume that we don’t need to do all of it however, commments on YT from a week ago show ‘thanks man, it worked’ so… yeah. Maybe I’ll just follow it to a tee:

sudo apt-get install libmtp-common mtp-tools libmtp-dev libmtp-runtime libmtp9

sudo apt-get dist-upgrade

sudo nano /etc/fuse.conf

lsusb

sudo nano /lib/udev/rules.d/69-mtp.rules

# Device
ATTR{idVendor}=="XXXX", ATTR{idProduct}=="YYYY", SYMLINK+="libmtp-%k", ENV{ID_MTP_DEVICE}="1", ENV{ID_MEDIA_PLAYER}="1"

sudo nano /etc/udev/rules.d/51-android.rules

ATTR{idVendor}==”XXXX”, ATTR{idProduct}==”YYYY”, MODE=”0666″

sudo service udev restart

sudo reboot

I’m going to walk through these commands one at a time and freshen them up if they need freshining.

sudo apt install libmtp-common <– not needed. shows current
sudo apt install libmtp-dev <– yes, required, not currently installed
sudo apt install libmtp-runtime <– no, not required, showing current
sudo apt install libmtp9 <– no, not required, shows current.

Therefore, the first line could be:

sudo apt install libmtp-dev

I’m a little concerned about running sudo apt dist-upgrade because I think that upgrades your entire current distro from, say 18.04 to 18.10… that seems a bit intense so I’m going to try to skip that one. Instead I did:

sudo apt update and sudo apt upgrade

Since we already have mtp-tools installed and we know that lsusb is showing the right stuff “Bus 002 Device 036: ID 04e8:6860 Samsung Electronics Co., Ltd Galaxy (MTP)” i think we can move on to adding the rules

blast. it looks like this one rule location might be out of date sudo nano /etc/udev/rules.d/51-android.rules

cd /etc/udev/rules.d/ to search for it…

not there… maybe needs reboot. Rebooting….

sudo nano /etc/fuse.conf
remove comment (#) ‘user_allow_other’
control x to save changes

sudo nano /lib/udev/rules.d/69-libmtp.rules <– note, this is different from tutorial, couldn’t find his file name and assumed this is updated name
1m15s video is useful to watch as he pastes

connect phone
do an lsusb in one terminal
open new terminal window so you can see both together

take this, and update the XXXX to your device things as listed in lsusb in the other window

# Device
ATTR{idVendor}=="XXXX", ATTR{idProduct}=="YYYY", SYMLINK+="libmtp-%k", ENV{ID_MTP_DEVICE}="1", ENV{ID_MEDIA_PLAYER}="1"

mine will become, for example AS DEMONSTRACTED at 1m44s in video with blue and red arrows (great visual, by the way!)
idVendor is the first 4 characters before the colon
idProduct are the second 4 characters after the colon.

# SAMSUNG GALAXY 7
ATTR{idVendor}=="04e8", ATTR{idProduct}=="6860", SYMLINK+="libmtp-%k", ENV{ID_MTP_DEVICE}="1", ENV{ID_MEDIA_PLAYER}="1"

I like to do stuff in text editors and make sure it’s good before pasting into terminals but up to you.

Oh CRAZY. it waslready in this sudo nano /lib/udev/rules.d/69-libmtp.rules

now what… hmm

well I guess we can say that was ‘step 1 to make sure that your device is recognized by MTP’. Now that is good we must do next stuff

sudo nano /etc/udev/rules.d/51-android.rules

not there. but foudn it here:

sudo /lib/udev/rules.d

so:

`/lib/udev/rules.d/51-

well that was all a waste of time, kind of. Turns out that the SD card is the mount point of the EMMC memory anyways (apparently) so I discovered there was an SD card in device and may work….

Categories
Freedom and Privacy Tutorial Ubuntu Touch

How to set up Gitlab with Ubuntu Touch 2FA Manager App

I had been using Ubuntu Touch Authenticator app, but I wanted to try out the 2FA Manager app to see how that worked for me. The only problem was that I had never reset my 2FA stuff that I originally set up so this was my first time switching devices. It was easy, but it was also not documented. I hope this simple guide helps and perhaps the steps of this process also apply to other 2FA sites:

In your Gitlab account

  1. Log into your gitlab account
  2. Go to “Settings” under your top right avatar
  3. Go to “Account”
  4. Click the ‘Manage Two Factor Authentication” button
  5. Click the ‘Disable two-factor authentication’ button which will then warn give you this warning “Are you sure? This will invalidate your registered applications and U2F devices.”
  6. Select ‘Ok’ on warning screen

This will now present you with a screen and QR code where you will scan it and bring it into the Ubuntu Touch 2FA Manager app. So now let’s move to your ubuntu touch device…

On your Ubuntu Touch device

  1. Download / install tagger app if you have not already done so
  2. Download install the 2FA Manager app if you have not already done so
  3. Open the 2FA Manager App
  4. Open Tagger app and point camera at QR code and patiently wait… adjust camera position if needed until it registers the QR code. You will know this is working when the dialogue with “open url, copy to clip, generate qr code” buttons pops up
  5. Select ‘open URL’
  6. Add a description. This is just a friendly name that will help you remember what this key is for as you may build up many keys over the years.
  7. Press ‘Add’ button. You should now be taken to the 2FA Manager app to a screen with a 6 digit bold code and a purple progress meter that is moving. You will notice that each time the progress meter reaches the end, it will loop back with a new 6 digit number. You will also notice that if you tap this 6 digit code it will open it up and make it much bigger and easier to read and also give you the ability to copy this code to clipboard. Whichever way you prefer, these are the two ways you can obtain your 2FA codes and these codes are what you will use to log into Gitlab each time, and also to use to finish up your setup in your Gitlab account.
  8. Go back to Gitlab

Meanwhile, back at Gitlab…

  1. Look at your 2FA Manager app and grab the 6 digit code presented on the screen. But hurry! (dramatic music here) If you don’t grab it quick enough the next one will be generated. This is for security if you were wondering and limits the chance of someone stealing your stuff…
  2. Enter the 6 digit code into the ‘pin code’ field on your Gitlab 2FA setup page you should still be on
  3. Select ‘Register with two-factor app’ button
  4. Save your recovery codes: If all goes well you should now be presented with a ‘congratulations’ screen and a list of 10 random looking letters and number strings with bullet points. These are your recovery keys so save them somewhere safe. I recommend the keypassxc app as it’s a really great password manager for your laptop / desktop ubuntu environment.
  5. After you have these recovery codes safely stored, press ‘proceed’. If you got an ‘invalid code’ message, it probably means you weren’t fast enough with steps 1-3 above so repeat but faster 😉

You should now be brought back to your gitlab Account page and good to go. You can probably use this workflow above to do 2FA setups in other systems as well.

Categories
Tutorial Ubuntu Touch

Flashing Ubuntu Touch to your Pinephone’s EMMC memory

You might have already read my first round of ‘flashing 101’ when I successfully flashed Ubuntu Touch to and SD card making it possible to boot the Pinephone that way with Ubuntu Touch. That first blog is a bit more messy because I was learning so in fact, this post here might also help you with flashing to an SD card as it should be more clearly written. If I had more time I’d clean them both up but I don’t so yeah…

The next step in my learning is to figure out how to flash / copy ubuntu touch onto the pinephone’s EMMC (built in) memory. This should make it run faster. There wasn’t a step-by-step that I could find and so I guess I have to write it. Nice folks in the pinephone telegram group said that I would follow a similar method as flashing to the SD card (see my detailed step-by-step on that here so that hopefully is true. Let’s try…

1. Plug in your Pinephone

Unlike my other tutorial, you’ll need the device plugged into your production machine from where the image will be coming.

2. Get latest image

Choose the latest Pinephone image from this page. I choose stable usually but as of today ‘stable’ is a figment of our imagination 🙂 Save it somewhere memorable and probably smart to put it in a dedicated directory so you can run commands more simply and safely. I just realized there is a bmap file sitting with this image download on this page. This could be useful and relevant soon as you read on…

3. Get setup with BMAP

You can just follow my detailed blog if you’d like. If you don’t know what it is, then you probably want to do this.

4. Confirm the source of your ubuntu-touch-pinephone.img.xz file

The file you downloaded in step 2 above which you have on your computer. In my case it’s going to flash from my download directory in my dedicated ‘pinephone_flashing’ directory so mine looks like this:

~/Downloads/pinephone_flashing/ubuntu-touch-pinephone.img.xz

5. Put Pinephone into ‘Yumi Mode’

I didn’t know what else to call this so I named it ‘Yumi Mode’ since that’s the name of the UBports Robot Mascot… I tried to simply plug in the pinephone and do this work but it won’t recognize anything. Thankfully I had read some other tutorial about how to get into this mode. Here’s how

a) Assure device is powered off (you can push / hold power button for about 5 seconds)
b) Press / hold power and volume up buttons together for about 6 or 7 seconds.

If successful, you should start to see a trainload of partitions start to load in your file manager (Nautilus).

6. Identify the destination path to the EMMC

If you’ve done my other blog where you install Ubuntu Touch to the SD card this time we’ll do the same thing but point the destination to the EMMC memory.

  • Hit the super key to the left of the space bar
  • Type ‘disks’ to open the ‘Disks’ utility. You should now see two items related to UT in the left pane of the ‘Disks’ app:

‘Drive | JumpDriv e microSD’

and

‘Drive | JumpeDriv e eMMC’

  • Click the ‘Drive | JumpeDriv e eMMC’ on the left pane
  • Take the path from the ‘Device’ path. In my case it looks like this “Device dev/sdc” but yours can – and very possibly will be – slightly different in the last letter. You can actually just highlight it and copy it from there

path: /dev/sdc

This is a great way to visually identify and confirm which mount point is your eMMC which will help you quickly deal with unmounting stuff in the next section

7. Unmount the partitions on the eMMC drive

If the drive is ‘fully ejected’ by Ubuntu, you will probably get this error when you run the final flashing command:

bmaptool: ERROR: cannot open destination file ‘/dev/sdc’: [Errno 123] No medium found: ‘/dev/sdc’

A reboot of the device again and putting it into ‘Yumi Mode’ again might fix it for you if you are lucky – once this worked for me – but that was it. I do not believe this can be done reliably and thus you should proceed with the command line method. It’s really not that hard if you work together with the Disks utility.

The reason why the ‘eject’ button in File Manager (Nautilus) doesn’t work for this purpose is because ‘it uses udisks ctl which both unmounts and ejects the partition’. We only want unmount, not eject, you see. So, we must ‘terminally unmount’ these as follows:

First, run the mount command which will show you (truthfully) what drives are (actually) mounted on the system. For me these were all located at the very bottom of the terminal output list.

You can, however, in the Disks utility, click through each partition and see at the bottom ‘mounted’ or ‘not mounted` or ‘unkown’ and compare against the terminal output to gain further confidence that you are unmounting and flashing to the right place. The Pinephone image has a mixture of all of these, it seems, so probably better to just manually unmount everything with the terminal. It’s really easy once you identify the actually-mounted drives you just run:

sudo umount /dev/sdc2' sudo umount /dev/sdc4′
etc
etc

**Tip **: If while running the final flashing commands you get this error:

bmaptool: ERROR: the image file ‘/home/wt/Downloads/ubuntu-touch-pinephone.img.xz’ has size 826.3 MiB and it will not fit the block device ‘/dev/sdc3’ which has 1.0 KiB capacity

You are probably writing to a partition like I did instead of the actual device. You probably put in /dev/sdc3 instead of /dev/sdc as an example.

8. Run the flashing command

This assumes you have alread set up and read and understand my blog above.

Format:

sudo bmaptool copy --bmap ~/path/where/your/bmap/file/is/located /path/where/your/image/is/located /path/to/memory/device

My actual example:

sudo bmaptool copy --bmap ~/Downloads/pphone.bmap ~/Downloads/ubuntu-touch-pinephone.img.xz /dev/sdc

Worked for me! Hope it works for you too 🙂

Categories
Technology Tutorial Ubuntu Touch

How to do an Ubuntu Touch Video Screen Cast Recording

Making a video screen cast of your Ubuntu Touch (“UT”) device is a bit difficult as of the date of this blog post, however, thankfully, there is a way to do it with a bunch of scary terminal commands.

This is pretty technical, but is probably the only way (for now) to do this. Special thanks to Amolith who forwarded to Joe (“in here”) who then forwarded it to me. The power of free software communities at work!

First, I’ll just give you the commands in case you are already familiar enough with terminals and commands. Then, I will provide detailed how-to to follow.

Record a big raw video file in interesting .raw format

EDIT 20/06/21
This first command has been updated to reduce ‘terminal weirdness’. Now it works quietly in the terminal. Special thanks to Rodney for that excellent improvement.

adb exec-out timeout 120 mirscreencast -m /run/mir_socket --stdout --cap-interval 2 -s 384 640 > ubuntu_touch.raw

Convert it to ‘normal’ format (mp4) with ffmpeg tool

ffmpeg -f rawvideo -pix_fmt rgba -s:v 384x640 -i ubuntu_touch.raw -filter:v "setpts=2*PTS" ubuntu_touch.mp4

I plan to give a few more how-to details next by editing this blog post but for now I at least wanted it logged somewhere where myself and others can easily find it.

Enjoy

Tutorial – How to actually do it.

Before you begin

I would recommend creating a directory / folder just for doing this stuff. Then, you will navigate in your terminal to this directory. That makes sure that all the video is dumping in a place where you’ll be able to find it and work with it easier. Otherwise, your terminal might autonomously dump it an undesired location (your Home directory if you left everything default…). Once you have created this directory, navigate to it with the terminal’s ‘cd’ command. If you don’t know how to do that, take a few minutes to search and learn? It’s a good life skill and you can brag about it at the coffee machine or water cooler…

Now that you have the scary command line codes above ready to copy and paste, here’s how you do it and what to expect.

First, you’ll have to make sure you have adb setup on your ‘production machine’ (the machine that is recording the Ubuntu Touch output video. You should be ok to just do the usual sudo apt install adb and then perhaps you’ll need to be in ‘Developer mode’ to do all this. You might find that your system already has adb. If so, it will tell you that your version is up to date, etc and you can continue. If you need either of these or find it’s not working right, this blog has a lot of good info in one place. Most should be current still at the time of this post.

Next, plug in your UT device and get ready to record. Probably a quick test video is wise, by the way, before you start recording a long meaningful video.

Next, run the first command above by copying/pasting into your terminal. You will see this (or something very similar) in the terminal. Just pretend this means “your raw video is now recording to your production machine” because that’s what’s happening even though you can’t see it:

daemon not running; starting now at tcp:5037

daemon started successfully

When you are done recording, go back to your terminal and press ‘control C’ in to stop the process. You will see absolutely nothing except your ^C. Just pretend this means ‘Your raw video file has now stopped recording and is sitting in the directory where you started it”.

Finally, you’ll need to convert the raw video to a human-usable format – and probably mp4 is what you’re looking for which is why the above command is setup the way it is. Note that if you want to do any other million different things with your video work, you could study the power of ffmpeg and do whatever you like with formats, resolutions, etc, etc….

Now, in your terminal after you run the converting command, you are going to see a lot of ‘stuff’ in your terminal. I’m not going to even paste it here because there is so much. I’m also not going to pretend I know what any of of it means. Just be aware this is ‘normal’.

Once complete you should now have a .raw video and a .mp4 video in your screen casting directory.

Double click your .mp4 file to make sure it’s working and enjoy your new Ubuntu Touch video screen recording.

Categories
Tutorial Ubuntu Ubuntu Touch

Flashing Ubuntu Touch onto SD Card for Pinephone Braveheart Installation

EDIT 20/06/19
I have done a fairly extensive post about how to use BMAP properly so here is a link to my post. You should probably read this to know that the true benefits of speed from Bmap tools comes from following the instructions in my other blog. Otherwise, you might as well just use ‘dd’ or other flashing techniques. Bmap is indeed faster!

EDIT: 20/06/14

I tried with Etcher and had problems booting (not sure this was actually the cause) so now I’m trying with a tool recommended bmap-tools by someone in the Pinephone Telegram group. This tutorial includes instructions for getting set up and using this bmap tools tool.

This tutorial assumes you are running Ubuntu on your desktop/pc/laptop…

It’s a command line tool, it seems so yeah. No GUI. Hopefully my commands and instructions will remove the CLI mystique…

Apparently bmap-tools also automatically unzips/uncompresses your ubuntu-touch-pinephone.img.xz image as well, so you don’t have to do that step before.

  1. Get Ubuntu Touch image currently here
  2. Install to ubuntu: sudo apt install bmap-tools
  3. Confirm the source of your ubuntu-touch-pinephone.img.xz file which you have on your computer from this instructional page your source and destination addresses. In my case it’s coming from my download directory and going to my SD card so my source location looks like this:

~/Downloads/ubuntu-touch-pinephone.img.xz

  1. Confirm the destination path of your SD card. This step is important because if you goof this you risk (probably) writing your image to the wrong drive and killing it. How I do this is
    a) Hit the super key to the left of the space bar
    b) Type ‘disks’ to open the Disks utility
    c) Click my SD card on the left pane (graphically)
    d) Take the path from the ‘Device’ path. In my case it looks like this “Device dev/mmcblk0p1” You can actually just highlight it and copy it from there
  • path: /dev/mmcblk0

Important note on this last step: I had a lot of annoyances in the terminal related to ‘memory errors’ and it was because I had copied the entire item. In my case it looked like this: mmcblk0p1. Make sure that you do not have anything like ‘p1’ or ‘p2’ on the end of your destination path. These are for ‘partitions’ and is the incorrect path. should be just : /dev/mmcblk0not something like this –> /dev/mmcblk0p1. If you run it with the latter you will get those memory errors like me.

Just to leave it as reference the errors caused by the mistake above look like this:

bmaptool: info: no bmap given, copy entire image to '/dev/mmcblk0'
bmaptool: WARNING: failed to enable I/O optimization, expect suboptimal speed (reason: cannot switch to the 'noop' I/O scheduler: [Errno 22] Invalid argument)
/
bmaptool: info: synchronizing '/dev/mmcblk0'
bmaptool: info: copying time: 17m 13.1s, copying speed 13.5 MiB/sec
  1. Run the command
    Pre-cautionary step 1: remove all external drives before doing any of this.
    Pre-cautionary step 2: Other tutorials I read did not explain that you should unmount your drive before beginning. So in your Nautilus just go through and ‘eject’ anything showing up as ‘ejectable’.

As a reference, this tutorial was helpful for tool usage but still not quite clear enough so I’ll go even slower…

EDIT 20/06/14 – Apparently this next bit of my advice is bad and slows down the process. Apparently you should just make sure you run the command in the same directory as the image, and then it will work faster and without the need for the –nobmap tag. I will leave this here as a reference just in case you need it but as of today it was advised to not follow it:

—-

With bmaptool when you are doing straight flashing you can just add the --nobmap tag to the command, which I think means ‘just do this thing without routing through other specific instruction file’. So that’s what I’m going to do.

—-

I am still not 100% sure if this step is required, but it helped me so I’m leaving it as a reference: I also got some ‘no space left on device’ errors so I decided to first format the drive with Disk tools before trying again. I did this by opening the ‘Disks’ graphical utility again by clicking the SD card I wanted to format and then selecting ‘format disk’ from the top right hamburger menu. Takes about 2 seconds to wipe whatever is on there… after doing this step everything finally started working better. I created a linux ext4 partition next to see if that helped. I made the first partition with 21GB (for the image) and then a trailing 10GB for ‘whatever’. These were just choices I made thinking that I could maybe use the trailing 10GB for phone storage.

Here are, therefore, the steps, assuming you have done all of the above:

  1. Open the terminal
  2. Make sure you have navigated (with terminal commands) to the directory where your image is
  3. Type the following command replacing the source and path with whatever is correct from your device as per instructions above:

bmaptool -E copy /path/to/ubuntu-touch-pinephone.img.xz /path/to/SD/card

My example that finally worked looked like this for your reference:

sudo -E bmaptool copy ~/Downloads/ubuntu-touch-pinephone.img.xz /dev/mmcblk0

Boot it

Apparently just inserting the SD card and turning the power on should do it, but yeah. Depends on a ‘few things’ as to whether this will be your experience 😉

This was actually harder than I thought. I’m not sure if this was the cause of my pain but I’m going to list it anyways. I had to do about 100 reboots and it still wasn’t working. I couldn’t seem to get past the ubuntu purple splash startup screen. I ‘think’ I had interrupted a process the first time so i did all of the above steps againb but this time I held down the power button until the green indicator light came on and then everything started as expected and I was able to start using Ubuntu Touch. Members of the community indicated it may be related to a faulty build and to try other build numbers to see if that helps. We shall see.

Issues flashing cards

I had an interesting issue where I couldn’t unmount a card. System said ‘not mounted’ when I tried umount command. Amazingly, just a reboot of computer allowed the ‘Disks’ utility to start working again as normal. Others were reporting that one should consider using gparted tools instead of Disks utility just as another related tip if you are interested.

I hope this helps and have a great day

Resources

Pinephone images for Ubuntu Touch here

How to telnet into the device here

My bmap tools tutorial here

Categories
Software Development Technology Tutorial Ubuntu Ubuntu Touch

GETTING A QUASAR BUILD RUNNING AS APP ON UBUNTU TOUCH

Here is my journey to see if I can make a basic Quasar app work on Ubuntu Touch.
I should preface that I have never built an Ubuntu Touch app and I’m not even done my Quasar course
but there are enough people interested in this so I want to stop everything and try…

Quick Overview of Ubuntu Touch Apps and Quasar

To get familar with the Ubuntu Touch directory structure stuff, I git cloned this simple hymnal app since I knew it was mainly based on html website stuff and likely would not have complicated databae backend stuff going on. You may wish to do the same.

Build an Ubuntu Touch App with Clickable

What is clickable? It’s the thing Brian Douglass made, of course. And who is Brian Douglass Brian is just super great and one of the main/only reasons I started learning how to program. Thanks Brian again!!

Here is a link to UBport’s App Development for Ubuntu Touch

I will start by forcing myself to actually read the docs! Painful for a guy like me but I’ve learned that this is the right way to suffer less… I will start by reading all the links in the ‘Guides’ section on the link above.

In there, I found Lionel Duboeuf’s video on clickable. Nice! Thanks, Lionel (another great guy).

First, don’t rely on my commands here that I reference. Always go to the documentation page to make sure they are all updated. I’m just putting them here mainly for my own log.

Part of my journey was discovering that the clickable documentation assumes I would know what order to install dependent packages. Unfortunately that was not true so my journey was a bit painful. I’ll try to spare someone else this part of the journey. The docs instruct to install docker, adb, git and pip3 but it doesn’t say to make sure that you do these first and that the rest won’t work. At least this is what happened to me. So maybe just follow my order:

  1. Install adb (search it online but I recall it was just sudo apt install adb)
  2. Install git (probably sudo apt install git)
  3. Install pip3 (the python thingy that installs more python thingies) so that you can install Clickable. sudo apt install python3-pip should be the right command… but your terminal should tell you if I’m wrong, ha
  4. Install clickable: This page says everything. I’m waiting for an answer on why pip3 install method is recommendd over PPA method. I’m assuming it has to do with docker functionality? Anyway, I’m up and running with the following command so roll with this: pip3 install git+https://gitlab.com/clickable/clickable.git
  5. Install Docker: See dedicated instructions below
  6. Run clickable in the terminal but don’t do so until Docker is fully installed! It requires Docker!

Installing Docker

This part of the process, for me, required a dedicated section and a lot of learning. If you are already set up with Docker you can probably just skip all this. But for those like me who have never used it this section should save you a lot of pain.

Apparently you need the latest ‘engine’ for Docker and for Ubuntu that is found here. This part was pretty painful for me but it doesn’t need to be. I’ll leave a few of my long notes to scan through but you should be good to go to simply run:

sudo apt install docker.io

Feel free to run that and skip this next section if you want.

Installing Docker the Fast Way

As per above: sudo apt install docker.io will install it but it’s better to follow the docker instructions above.

Installing Docker the Long ‘Recommended way’

Setup some preliminary ‘stuff’ with this group of commands:

sudo apt install apt-transport-https ca-certificates curl gnupg-agent software-properties-common

Curl the gpg key with this command:

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

Looks like this next one sets up the right repository. Note! You can copy the back slashes and it works just find in terminal (somehow I never knew this…):

sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"`

Run this:

sudo apt-get update

And then this:
sudo apt-get install docker-ce docker-ce-cli containerd.io

Finally, run a quick test to make sure it works:

sudo docker run hello-world


EDIT – 200427 apparently this next user section is not needed as it happens when you install Docker. Thanks

But don’t think you’re ready to go just yet! You have to make sure your Ubuntu machine has the right privelidges set for Docker.

Add Docker User to Ubuntu Group

If you know how to ‘add Docker to your user group’ or to check to see if you have it, then go ahead and do that and skip this section. For the rest of us, read on:

Apparently you can see what groups and users you have on your ubuntu machine by using this handy command, but note importantly that the list it outputs is not in alphabetical order:

For users: compgen -u
For groups: compgen -g

If all the Docker install stuff worked you should see ‘docker’ in the list under after running compgen -g

If you don’t, then if you run the following command you can add it. Be sure to replace the $USER – incuding the $ with your Ubuntu machine’s user name.

sudo usermod -aG docker $USER


CLICKABLE TIME – THE MOMENT OF TRUTH

This section now assumes that you have done all the steps above including Docker. Here is a quick review so you can check again:

  1. adb
  2. git
  3. pip3
  4. clickable
  5. docker

Before you do this next step, if possible, make sure you are on a wired connection or the fastest connection you can get. I will explain…

Ok, let’s do this. Type clickable in your terminal.

Note! this part is wacky and doesn’t deliver the usual ‘status’ stuff to know what’s going on and how long it will take. There were a lot of the following kind of messages in the terminal which were normal. Here are some notes I took so that you can feel less ‘curious’ about whether things are progressing or broken:

waiting
Download complete
Verifying Checksum
etc
etc
Download complete

At this ‘Download complete’ message this is where things hung for a long time for me. Apparently what is happening here is that an entire Docker image is being downloaded and set up and this is nearly 1 gigabyte of data.

Pull complete

Once I hit this stage ‘Pull complete’, then there was another batch of messages that included ‘verifying checksum’ and ‘Download complete’ at which point it again paused for a really long time.

After what seemed like forever in my case (bad wifi) it all ended the process.

It’s also interesting to note, but I have not tested, that apparently this long process only happens the first time you set up your app and subsequent updates you make to your app will not require this process, which is nice.

Ubuntu Touch(ing) Your Quasar Package

Now you’ve powered through that process you’re now ready to move your Quasar app to Ubuntu Touch. The good news is that this part is pretty easy! I’ll try to make this more of a step-by-step guide since probably we’ll all need to refer back to it.

Setup your Quasar stuff for the Ubuntu Touch stuff

This assumes you know what quasar build means and that you have have already run it in your quasar directory and have the ‘app’ ready to go. On my machine, the files we need are found in the /dist/spa directory.

  1. Create a directory on your development machine with your Ubuntu Touch app name somewhere logical for later use
  2. In the same directory create another directory called ‘www’ (you can probably call it what you want but might want to keep it ‘www’ for this tutorial for ease). To be honest I think the main purpose of doing this is to keep your source files in a logical spot. At least that’s why I did it…
  3. Go to your Quasar /dist/spa directory and copy all the files from there to the ‘www’ directory you just created in step 2.
  4. Set up your logo file for UT. I think there are two ways to do that so I’ll outline them both with a and b as follows:
    a) stick it in the main directory of your directory you created in step 2 above and give it the title (“icon.png”)
    b) stick your logo file in a logical place then go in and edit your appname.desktop file and point the following line to your icon: Icon=path/to/your/icon.png

Now you have the foundation laid and are ready to build the app

Build the Ubuntu Touch App Pre-Build Structure

  1. In your terminal cd inside your app’s top level directory, run clickable create which will create the structure for your app, not the app itself (the click file). I was mixed up on this for a while so note this.
  2. As Clickable walks you through the process, the key step is to select option 5 (HTML) . The rest of the options are for standard app building processes and details about those can be found if you’d like to learn more or just watch Lionel walk through it in his video.

Move your Quasar files to your Ubuntu Touch app structure

  1. Manually copy/move all the contents of your top level app `www’ directory into the newly-created /YourAppName/www folder (I wonder if we can automate this step later so the build script grabs it?)
  2. Run clickable desktop which gives you a really cool test emulation of the app before you build it. If everything seems to open up and work on your dev machine, continue.
  3. Make sure your Ubuntu Touch device is plugged in by USB (so that it will get the adb push of the app) and recognized. I also discovered there is a handy clickable tool clickable devices to do just that.
  4. Run `clickable’ which will build the app, and push your Quasar app to Ubuntu Touch as a click file. You should now be able to find a newly-created /YourAppName/build directory which should hold your shiny new Quasar appready to publish to OpenStore, or use on your local Ubuntu Touch device for whatever you need.

Success? Let me know if it helped and how the tutorial can be improved as I had to rush it out for someone 🙂

Categories
Technology Tutorial Ubuntu Ubuntu Touch

Getting Started With Quasar and VueJS on Ubuntu

After watching this recommended tutorial I decided to try out this Quasar thing. My true hope is that we can eventually add Ubuntu Touch as one of the buildable outputs at the end of app creation as it apparently might not be that hard with Cordova… but I digress…

I’m not an experienced developer yet so getting started with all these different ‘things’ I had to install with the terminal was getting a bit concerning. I decided to document my journey so that others can hopefully follow along and save some research and time.

So, if your goal is to be able to, without a great deal of pain, create a good looking app to run on Ubuntu, this might just help.

Without further adoo here are the steps that I took to get set up on my Ubuntu 18 LTS machine so that I’m ready to start using the Quasar framework. I left very brief notes to show what you are actually doing so that it’s not some random terminal script from some random blog:

  1. Install npm
    sudo apt install npm
  2. Download and install NVM (a thing to help you install and manage nodejs. Ubuntu repository for NPM is apparently not reliably up to date so that’s why we are using NVM. NOTE: Before beginning, check this page to make sure the following script I’m about to give you is up to date as it changes from time to time. Oh, and if you want to have a dance party and learn more about the details of NVM, this video is good. curl -o https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh | bash
  3. Install LTS version of Nodejs (you can do whatever version you want as well easily but I’m not bothering to show that in this tutorial): nvm install --lts
  4. Check to make sure what you installed was what you wanted nvm ls
  5. Install the Vue CLI which is apparently what real developers developing VueJS stuff use. I got a few ‘warning’ messages after running this about my OS but probably life will continue…note also that when running this script my whole computer slowed down: sudo npm i -g @vue/cli
  6. Install Quasar – the framework that apparently makes doing a lot of stuff easier: sudo npm i -g @quasar/cli

Now I’m off to try to actually do this tutorial

Some Follow-Up Updates from the first tutorial Journey

Might as well leave a few notes that might help you along the way. If you were doing this Quasar tutorial and had an issue with hot reload not opening, I figured out how to get it working by just running quasar dev again and that seemed to kickstart the hot reload option. I had this happen a few times when i made changes and nothing new showed on the hot reload app.

In order to get highlights working properly in VS Code like teacher’s you should get the extensions for Vue and Sass since they may not be installed. I used:

octref.vetur (Vetur) for Vue because this has html auto-complete as well (another one I tried didn’t…) and

syler.sass-indented for sass stuff

Converting your New App into an Electron App and then Building for Ubuntu with a Debian package

For some reason it doesn’t really say explicitly anywhere in the tutorial above nor on the Quasar site, how to build for Ubuntu / Linux. I assume most developers ‘just know’ this but I didn’t so I had to search and find. For your benefit,

Installing for Ubuntu

Very hard to figure this out for a noob but after a full day of time spent I figured out how to turn this Electron app into an ubuntu native app running in the launcher and everything (exciting moment for a noob, I say!)

I feel like there should be a way to do this right in Quasar with a terminal tag during the build but it seems not.

First, you need to install this by running this on your Quasar dev machine:

npm install -g electron-installer-debian

Then, you build the app with the Quasar electron instructions, but for me it was as simple as running this:

quasar build - m electron

After you run that you’ll find two directories, one of which I still don’t know what it does. The two directories were “my-app-name-linux-x64” and “Unpackaged”

I then created a directory in my quasar project folder at the top level called ‘installers’

I then took the following command which I pulled off the read me page above:

electron-installer-debian --src dist/app-linux-x64/ --dest dist/installers/ --arch amd64

And adjusted the ‘src’ (source) to point to my app-named directory in the electron folder and the ‘dest’ (destination) to the installer directory I created. You can do what you’d like but mine looked like this therefore:

electron-installer-debian --src dist/electron/my-app-name-linux-x64/ --dest installers/ --arch amd64

The result was a fully effective .deb installer file which installed without a hitch and really ended the above tutorial on a high note.

Important note: I had this error:

Creating package (this may take a while)
Error: could not find the Electron app binary at “dist/electron/Quasar App-linux-x64/my-app-name”. You may need to re-bundle the app using Electron Packager’s “executableName” option.

This took a long while to trouble shoot but the issue was nothing more than opening the package.json file and making sure that the following two entries match and probably have dashes in the names (haven’t confirmed that need but it worked with them so I’m sticking to it!)

“name”: “my-app”,
“productName”: “my-app”,

Hope this also saves someone some hair pulling

Cordova Stuff and Ubuntu Touch (raw notes for later)

Cordova is apparently our best hope to get Quasar stuff to be able to convert easily to the Ubuntu Touch framework so I’ll leave my quick notes although they are quick, dirty, and probably fairly useless. I don’t think it can support Electron as per above.

First, apparently… cordova was not installed yet when I ran quasar build -m cordova it never told me that… I had to install cordova with some command that I cannot seem to now find…

Also interesting is that the build let’s you build without error even if you have caps problem in command such as typing ‘Cordova’ instead of ‘cordova’. Neato.

A little late in the game this link was sent to me which I somehow didn’t find before beginning even though I searched. I would have started right here so hopefully this documentation link helps someone save some time.

More to follow, hopefully…

Categories
Freedom and Privacy Life Skills Nextcloud Tutorial Ubuntu Touch

How to Set up a Calendar with Ubuntu Touch and Nextcloud

1. Set up online account in Ubuntu Touch

You will need all your calendar info from your Nextcloud instance before beginning. Looks something like this: https://yourdomain.com/nextcloudserverlocation/remote.php/dav/calendars/username

  1. open calendar
  2. middle icon that shows grid calendar
  3. ‘add online calendar’
  4. choose NC
  5. enter credentials

2. Sync

UT calendar ‘should’ start syncing right away and you’ll see a little icon appear showing that sync started and stopped. Sometimes, however, it seems like a manual sync is required. You can do this by going to the calendar page and selecting the top right hamburger menu and then the circular arrow ‘refresh’ icon.