How to Do Some Code Changes in Gitlab
I’m not a developer but was encouraged to try a one character change to some software. I felt that I couldn’t break too much so I did my first tiny change. Literally one character text edit. However, it was a bit scary so I’m just posting this to show the basic workflow. This assumes you already have a gitlab account and username and can log in.
1. Find it.
probably you found or someone sent you a link to where the code is and you are on the gitlab page and can see the line of code you’d like to change. If not, get there.
Also, make sure you’re on the right branch. I tried to click ‘edit’ but I was not allowed because I was not on the branch called ‘master’. Your branch that you can edit may have a different name but just be aware that you have to be on the authorized branch that accepts changes. the left side has ‘branches’ and you can look at them there and select the right one.
2. Fork it
When you click ‘edit’ it seems to automatically force you to ‘fork’ the code. This makes sense. You wouldn’t want to disrupt the main code until the overseeing person has reviewed your changes and authorized them. So you ‘fork’ it and do your changes there. I think a fork is basically a copy/paste of the whole block of code for that file
3. Change it.
On the page where you change there is a box where you can comment. I was a bit worried about this but it wasn’t so bad. it’s just a comment box but it seems a bit more important than that. I put my personal comment about the change I made and then saved it.
4. Merge it.
Once you save the changes it becomes a ‘Merge Request’ also known as MR.
I think this part depends on how the project is set up. It might merge automatically into the main software or, it might require a ‘stamp of approval’. Regardless, once you send the MR it’s out of your hands unless you are the developer overseeing the project, also known as the ‘maintainer’.
Hopefully this helps someone else make their first change and thanks to the folks in free software who helped me do this!
THE NEED FOR A NEW QR CODE WORLD – OR SOMETHING LIKE THAT…
The Background – The Feelings
The background to this one is that I was really amazed by the technology and functionality of QR codes. QR codes brought the convenience of UPC retail bar codes to the average person without the crazy costs associated. So, even from a business perspective you could have your own scanning system if you wanted.
On a personal level, it was mind blowing that I could have a business card with a QR code on it that would ‘hide’ data within visually and then the smart phone or web cam could see the code, unpack the data and leave it with you in a digital format – with no typing needed and no errors (unless the creator goofed). This allowed me to turn a business card into an immediately usable tool, instead of what would happen before where the paper card would get buried or lost before I got to use it, plus it would force me to type more keystrokes than I wanted. I mean.. It could even do a .vcf format something which could be loaded direct into your contact books, from what I recall (would need to review this one but 99% sure I did that…)
Further to the personal benefits, let’s say you are at a social gathering. Someone approaches you and wishes to get your business card, or, some form of contact informaton that might normally be on a business card. But, you don’t really want to share certain parts, or any parts with said person. These days normally you would fumble and figure it out on the fly. What if the person says ‘text me’? Typically I say “I don’t text” and then I have to have a long discussion about why I don’t text and how to reach me, and here’s my email, and here’s my number, but don’t call me, because I don’t like phone calls, blah blah blah… It would be nice to have a standardized way of transmitting data – line of sight – no internet – no wifi – no bluetooth. Just transmit basic data. No worrying about whether they are on apple, or android or Ubuntu Touch (wait, that’s no problem!) or whatever. No worrying about whether they have a cell phone or a cell phone plan, etc. It’s also more inclusive that way.
I can hear the voices now:
- “But you can email all that!” – No! I want faster. No typing. No thinking.
- “But you can text all that!” – No! I said I don’t text. Don’t ask me to text you.
- “But you can ‘bump’ with Android” – I don’t bump. I don’t Android.
- “This is making something new that we don’t need!” – Ok… maybe… And that’s the point of this blog. Let’s figure that out…
Why QR has failed thus far (In my opinion)
After all these thoughts above, I got to thinking about the limitations of QR. You can read some of my feelings in this bug report I filed at UBports for the Camera app. Of course, I completely understand the security need here and I can’t argue it. But this has always been the issue. For QR codes to flourish, the scanner must be one button away.
And, even if you had one button there is still the annoyance of having to focus the camera. I thought about creating a tiny keychain which had one button and a camera. You would point at the QR and push/hold the ‘shutter’ button. It would focus, scan the QR code and beep when it was logged. Later you could sync that data with whatever device you want, somehow securely.
And that’s when I started thinking that it would be cool to have an ‘audio QR code’. So I searched that and found Chirp. They basically have figured it out with proprietary software – not typically a good solution if you want awesome security and privacy… But, their stuff apparently works. It sends hypersonic sounds and data.
In my opinion, this direction could work, and here are the first pros/cons:
Pros
- No focusing of cameras (faster scan?)
- No camera at all (cheaper hardware?)
- Audio? Maybe not as radioactive as radio??? (just throwing it out there for the healthy people?)
- Other cool pros?
Cons
- Apparently dogs don’t like hypersonic?
- Would these things have to transmit all the time or could the ‘chirp’ be triggered with some kind of ‘hey! any birds out there who want to chirp me anything?”
- Security? Could anyone chirp anything? I haven’t really looked deeply at that part yet as you can see.
Some suggestions to the conversation were:
- A watch (with a cam – full Dick Tracey style)
Sure, except I would never wear a watch, and I’m not a Dick…Tracey… - Near Field Communication (NFC) which is explained well in this old 2014 vid
Then, whatever the solution is, would be able to securely/safely transmit the stored data into the device of your choice for reviewing, handling.
Final thoughts
I feel like I don’t want to add to the smartphone in my pocket, but, it seems like there is no other way. Security people say you can’t really have a one-button something without compromising security of smartphone. SO…
That’s what I’ve got to say.
I hope some kind of cheap and useful solution lands here…
CONVERT HARVESTED BULK EMAILS INTO USABLE SPREADSHEET IN A FEW STEPS
Did someone send you a bulk email and foolishly leave all the recipients exposed? Do you want to grab those recipients and use for your own great purposes? Good news, it’s not so hard to do!
Here’s what you’ll need:
- A text editor (gedit on ubuntu)
- Spreadsheet software (Libre Office is the one I’ll use here since it’s both free and awesome)
1. Copy and paste the emails into a text editor
Grab the group of email addresses starting from right after the ‘to: ‘ but don’t actually include the ‘to:’. Copy this into your clipgboard.
2. Paste this to a fresh text editor window
3. Clean up the Paste
Make sure there is a trailing semi-colon after the last entry and no other gobbly gook before or after, other than just the names of the recipients and the actual email addresses. So like this stuff:
John Doe johndoe@johndoe.com; Jane Doe janedoe@janedoe.com;
4. Save file as a .csv file
Click ‘save as’ and give the file a handy name and make sure it has a trailing .csv as the file type
5. Go and open the file.
In my case it opens the file with LibreOffice – this is good. In your case…how could I know your case?
6. Select ‘semi-colon’ as the separation type
When Libre Office opens the file, uncheck comma, etc, and just leave the semi-colon because that’s what you’ve got.
7. Reformat the Orientation (optional)
After importing thus far, Libre Office seems to make the orientation horizontal putting one email address in each column and all the entries into a row spanning many horizontal columns which I didn’t like. I want to convert this to a vertical orientation with the entries spanning downwards across many rows.
Yippee yay. Someone alrady figured this out here
But in short, just do this:
- copy all your horizontal entries
- copy them to clipboard (control c)
- control + shift + V into the cell you want
- Paste (which does the ‘paste special’ command)
- Delete the original row you just copied from
Voila. Clean vertical list of harvested emails!
Hope that helps.
My Comments
My only improvement would be to add a script that pulls the name out and adds to the column on the right. I know thunderbird can do this when you right click and either ‘copy email address’ or ‘copy name and email address’. So it’s definitely possible to do this. In fact, how cool would that be if Thunderbird could just do all this in an add-on … hmm. Too bad I can’t program…
How to Install Video Convertor App on Nextcloud
Someone complaining that they can’t view an Apple .MOV file on your Nextcloud server? No surprise considering ‘apple is apple’.
Heard about the cool app ‘Video Convertor’ for Nextcloud? Tried to install it and use it and got that ‘requires ffmpeg error?
Same.
Turns out it’s pretty easy to to get it going and here is how:
Let’s do it.
- ssh into your Nextcloud box. In my case it’s a nextcloudpi box if that matters…
- type this into the command line:
sudo apt install ffmpeg - Hit your enter key
- Wait a long time while a seriously large amount of stuff installs (and probably this is why it’s not installed by default)
- Go back to your video file in the Nextcloud in the browser user interface and note the three dots on the right side of the file name. Click that
- Select the video icon ‘convert into’
- Choose your options for speed and format
- Do it.
- Wait again (a lot of waiting here…)
- Create a new share of the newly created video file
- Test the file (good practice) by sending the share to yourself in a browser that you don’t use so you are seeing what your recipient will see without your login credentials affecting things.
- All good? Ship the new link to your recipient
Hope that helps
INSTALLING ROUNDCUBE 1.4 VERSION ON YUNOHOST
Thanks to the work of Brian we can now install the newest version of Roundcube on Yunohost.
Why is this so exciting?
- Newer, fresher UI
- PGP encryption Functionality with the enigma plugin
- Mobile friendly skin
In short, it makes your self-hosted email awesome on a mobile too, regardless of whether you have an email app that works or not with PGP
Normally, if this app was an approved app in the Yunohost app list you would be able to simply search it and install it from the app list. Until then we have to install it by the command line but it’s not that scary at all so let’s begin.
Installing it
- ssh into your Yunohost box
- run this command to install it:
sudo yunohost app install https://github.com/bhdouglass/roundcube_ynh/tree/testing
Configuring the Install
A bunch of questions will start. Here is how I answered mine but you can adjust as you like. Note, if you press the enter key it will choose the default option for quick installation. Critical step is choosing ‘yes’ when you are presented with the Enigma opion.
This first question is just a warning. You’re brave. Take that risk!
WARNING! Installing 3rd party applications may compromise the integrity and security of your system. You should > probably NOT install it unless you know what you are doing. Are you willing to take that risk? [Y/N] : y
This next step shows all the domains you have configured in your Yunohost box and will ask you which one you want this Roundcube to be associated with:
Available domains:
-domainone.com
-domaintwo.comChoose a domain for Roundcube (default: domainone.com:
This next question lets you choose which URL folder you want for the mail. I changed mine from the default since I have other things running but you can leave default if you don’t have something already using ‘/webmail’:
Choose a path for Roundcube (default: /webmail): /pgpmail
I’m using Nextcloud calendar stuff so i don’t need CardDav stuff now but feel free to install it if you need it and want to use it:
Install CardDAV synchronization plugin? [yes | no] (default: no):
This is an important one if you plan to encrypt your emails!
Install Enigma messages encryption plugin? [yes | no] (default: no): yes
Testing it out
Once complete, you should now be able to go to ‘domainone.com/pgpmail’ (or whatever your options are) and hit the new Roundcube installation and log in with your Yunhohost email server settings. I recommend doing the following tests each time you get setup with something like this since desktop browser and mobile browser use different skins (sometimes) in Roundcube.
Before beginning, make sure that the public key of each email address has been sent/imported into each side of the email transaction. You can use the ‘import key’ feature in Roundcube’s desktop mode quite easily. At point of this tutorial I haven’t tried importing keys with Mobile browser, so I can’t confirm if that works or works well.
- Send totally plain text test message from desktop browser to test email address
- Send totally plain text test message from Mobile browser to test email address
All good? Now with public keys attached:
- Send new email with just public key attached using Roundcube’s ‘attach public key’ feature – from Desktop browser
- Send new email with just public key attached using Roundcube’s ‘attach public key’ feature – from Mobile browser
Still good? Now encrypt it!
- Send new email fully encrypted using Roundcube’s ‘encrypt this email’ feature – from Desktop browser
- Send new email with just public key attached using Roundcube’s ‘encrypt this email’ feature – from Mobile browser
Everything still good? You should be ‘in business’
Hope you enjoyed and found this useful.
HOW TO INSTALL NEW ROUNDCUBE VERSION ON SHARED HOST CPANEL WITH PGP KEYS WORKING
So, you want to not wait for RoundCube to release 1.4 to cpanel, or, you have Roundcube on your cpanel setup and for whatever reason the Enigma plugin that makes the PGP stuff work – isn’t working. Whatever your reason is, the solution is not insanely hard, but it took me about a week and a lot of hours to figure out how to get it going. Hopefully this will save you many hours that I lost! 🙂
Probably this page will work for future stuff too for future releases so I’ll leave the download pages more general.
Before we begin, quick thanks to everyone at Roundcube for really improving the look, feel and security of everything. Really nice upgrades in the UI for version 1.4 RC!
STEP ONE – DOWNLOAD
- Get your version (in this case 1.4-rc) from this Roundcube download page
- Download compressed Roundcube file to your computer
- Upload compressed Roundcube it via FTP (or whatever method you like)
- In your cpanel File Manager, Right click on the compressed Roundcube file and ‘extract’ – the file will decompress and extract the directory with the same name in the same directory where you clicked ‘extract’
- Locate the newly extracted file and rename to something you like. This will form part of the URL when you log into your email. If you leave it as is the URL will be too long and annoying so changing it to something like ‘mail’ or ’roundcube’ would be smart. Maybe don’t call it ‘webmail’ because most shared hosting uses that? Not sure, I didn’t test that but something more unique would probably be smart.
- go to the URL of your domain, with the new directory folder and add ‘installer’ at the end and this will start installer. You can read details on the Roundcube installation page and we are now at step ‘Configuring Roundcube’
Create your MYSQL database for Roundcube
You’ll need a database to make Roundcube work. Although this kind of stuff seems scary, it’s not that scary as long as you don’t delete stuff that’s already there that you don’t know about. In short, all you are gong to do is create a database, create a user for the database, and then glue the user to the database. That’s it.
Before you begin this part you should have some kind of password manager software setup, I think, so you can create strong passwords and not lose them. Otherwise, ‘do it the way you like’.
Here’s how:
- in your Cpanel home panel, go to ‘MYSQL Databases’
- in the top section, Create New Database, give your new database a decent name like ’roundcube_abc’ (no one sees this stuff, it’s just back end). You will note that it will automatically append something to the front. That will be part ofyour database name, not just the part you are typing in the field.
- Click ‘Create Database’ button
- Scroll down to the MYSQL Users section and in the ‘Add New User’ section at the top, create a new username with a strong password. You can use the password generator and then make sure to save it safely. You will note that it will automatically append something to the front. That will be part ofyour username name, not just the part you are typing in the field.
- Click ‘Create user’ button
Now you have both a database and a username. the last steps are to glue them together.
- Scroll down to ‘Add User to Database’
- From the ‘User’ dropdown, select the user you just created
- From the Database dropdown, select the database you just created above
- Click the ‘Add’ button
It will take you to another screen where it asks what permissions you want to give this user in this database. You will give it all permissions which should be the default (all boxes selected).
Confirm these changes.
Installing Roundcube
Really you should read this entire page and learn a bit as your setup might be different and it’s good to review the items they mention, although most or much of it is for people who have full access to their servers, not cpanel people.
Once you start the installation process, there really isn’t too much documentation on how to actually set it up. It will start a kind of ‘installation wizard’ but not a lot of help is there during the process. You’ll need to have the following items near you before you begin:
- The email credentials that will be using Roundcube including: server names, type of email (ie. SMTP, IMAP), security protocol of each, ports for each. You can get all this from your email provider.
- The database name, user name, and user name password for the MYSQL stuff you created above
Once all this is in hand simply walk through all the fields and do your best. The details of that part is out of scope of this tutorial but there is one vital point which you absolutely must seclect in order to use encryption and PGP and that is, in the plugin section you must select the Enigma plugin which will add all the functionality.
Again, be sure to install the the Enigma plugin.
Fixing the enigma_pgp_homedir not specified error
After installation is complete and everything appears working, if you were to go to the settings and to PGP keys you will be greeted with a warning that enigma_pgp_homedir is not specified, or, if you tried to simply put the path in there as if it were a full control server, it probably won’t work. At least that’s what happened to me and the purpose of this tutorial.
The solution to properly point Roundcube to a secure folder was pretty hard to figure out but now that it’s figured out should be pretty easy for you.
You should apparently not put your pgp key folder into the enigma plugin folder, or in the document root folder. Frankly I”m not skilled enough to know all the whys but I’ve learned to trust people smarter than I. So, I put the directory in the main directory of my domain where roundcube is. You can mirror this for simplicity and probably should. So, here go the steps:
- In cpanel File manager go to: public_html/yourdomain.com
- Create a folder for your keys. For this tutorial I’ll call it ‘keez’ but you can call it whatever you want. Make sure permissions are 0755 (they should be after you create it)
- In File manager, navigate to this location: /yourRoundCubeInstallationFolder//plugins/enigma/
- Locate the config.inc.php.dist file
- Touch it, right click on it, and rename and remove the ‘.dist’ from the end of the filename and save the change
- Touch the newly renamed file again and right click and ‘edit’. This will open the Cpanel text editor.
- It will give you a warning that by editing you can break everything, which of course we know, so, click edit again and reall stick it to the man!
- Scroll down in the code stuff until you see this section:
// REQUIRED! Keys directory for all users.
// Must be writeable by PHP process, and not in the web server document root
$config[‘enigma_pgp_homedir’] = null;
Now is the fun part. All you have to do is enter in the path that points to your keys folder that you created in step 2 above.
Mine now looks like this:
$config['enigma_pgp_homedir'] = realpath(__DIR__.'/..'.'/..'.'/..').'/keez';
As long as you put your keys folder in the main directory of your domain and named it ‘keez’ this line shoud work. If you have your folder somewhere else you’ll have to adjust accordingly. What I learned was each instance of '/..' moves up one level directory. So this command is saying ‘you will find the ‘keez’ folder by going up 3 levels from where you are now”.
Once you have adjusted this one line of code, click the ‘save changes’ button in your cpanel editor.
Go to your Roundcube email, go to ‘settings’ and go to ‘PGP Keys’ and you should now have working PGP functionality working in your Cpanel Roundcube, plus, you are running the new mobile-friendly version 1.4 now that all the paths are working.
I hope this helps!
HOW TO MAKE A NEXTCLOUD PI BOX WORK AS REVERSE PROXY TO YUNOHOST
Background
The situation was that I wanted to test out the very cool project Yunohost but I already had Nextcloudpi (another awesome project!) running on my local network. I already had a DDNS service (No-ip) running which was pointing to my Nextcloudpi (“NCP” moving forward) box, and a second DNS service that I set up which pointed to my router for the purpose of Yunohost (“YH” moving forward). You can read about that cool DNS solution in my other blog post, by the way, as it works really well and gives a bit more power.. and it’s free.
The problem was that ports 443 and 80 were being used by NCP but YH needed them as well. The only options appeared to be:
a) change the ports of one of the machines (complicated because clients outside of the LAN in the world webs won’t know those ports) or
b) figure out what a ‘reverse proxy’ is and then make it work
The challenge was that NCP was using Apache whilst YH uses NGINX – both of which are capable of reverse proxy. So, in order to do this I ended up doing some learning of both although it turns out it wasn’t really needed after all. C’est la vie…at least I learned some things!
At the end of the journey of trying about 10,000 different settings in the Apache default configuration file that comes with NCP (and other Apache installs) called “000-default.conf” it started working after adding just two lines to my configuration which seemed not to be in any other tutorial online for some reason. The key two lines that were needed were:
SSLEngine On
SSLProxyEngine On
Without those two lines it would just never work even though the rest of my settings were right.
Ok, enough of my hard journey story, let’s log the actual configuration and steps so that anyone who wants to do the same setup can save the pain!
Assumptions
Before we begin, I will assume that you already have the following set up:
- Server A (in my case NCP) running Apache which is already successfully reachable and working from the outside world. Through this machine Server B will be reached.
- Server B (in my case YH) running whatever (I think) but in my case it’s running NGINX and this box is the one we are trying to make visible to the outside world through ports 80 and 443
- You have a domain (nameofyourdomain.com in this tutorial) which you own and which is already successfully hitting your router (You can test by pinging the domain and seeing the IP address of your router show up). You can do this with my other tutorial mentioned above as well. You can also get a free ‘domain’ from services like No-ip if you don’t care what the domain looks like.
- You have full access to SSH into both machines, but in this case Server A is the critical one.
- You are using an Ubuntu environment and have know how to open a Terminal and use it (roughly)
- You are willing to learn and try things if this doesn’t perfectly work as per this specific example. I’ll give you a few resource links as well to help you in case your set up needs tweaking.
Let’s Begin – Setting up Apache Default Config on Server A
- ssh into Server A (format ssh username@your.IP.Address )
- Change directory (cd) to your Apache2 sites-available directory. In my case it looks like this but if you aren’t using NCP it might be different
cd /etc/apache2/sites-available - Type this command to back up your Server A apache settings. If you mess anything up you can restore this one and delete the default and rename it back to original name.
sudo cp 000-default.conf 000-default.backup
- Check to make sure the new file with .backup is showing up by typing ‘ls’. If it’s there then proceed.
- Copy the sample configuration below into your clipboard
- Open the default Apache config file with this command (if you haven’t used nano before probably good to do a quick online overview) for editting:
sudo nano 000-default-conf - you may have some settings already in this file (you should) at the top. Scroll down to the bottom of whatever is there and then paste in the sample you have copied from below with the control + shift + v (If you don’t hold shift it won’t paste)
- Go through the newly-pasted configs and adjust to your settings changing domain names and ip addresses to yours.
- Control x to save and exit, ‘y’ to save modified buffer and ‘enter’ key to write your changes
- Restart apache with this command to see if it works (this will shut down whatever stuff is running on Server A so probably good idea to do this wisely if the server is currently being used by others…:
sudo systemctl restart apache2
If you get nice silence from your terminal, and no ‘journalctl’ messages, then things are going the right direction.
Run Let’s Encrypt Manually for SSL certs on Server A
For this step, to be honest, I’m not sure if you need to do it because certs are already on both boxes for NCP and YH. But you might not have that so I’ll provide the steps since after I did them nothing was worse and everything was working… I would love to get some feedback on this step.
- Install Let’s Encrypt tools:
sudo apt-get install python-certbot-apache - Run it
sudo certbot --apache -d example.com -d www.example.com
Let’s Finish – Test Server B
Go to your domain from outside your LAN (just to make sure you are getting a real test) and try to hit Server B. I find mobile phone data plans are good for this kind of testing, otherwise, call your grandma and ask her what happens when she goes to nameofyourdomain.com…
If it works, you’re done.
If it doesn’t you might need to tweak your settings.
Sample Configuration – copy this and adjust to your set up
Your IP address will obviously be changed to the correct one where your Server B is. Copy everything in the code block below.
<VirtualHost *:80>
ServerAdmin name@nameofyourdomain.com
ServerName nameofyourdomain.com
ServerAlias www.nameofyourdomain.com
ProxyPreserveHost on
ProxyPass / http://192.168.1.37:80/
ProxyPassReverse / http://192.168.1.37:80/
</VirtualHost>
#Listen 443
<VirtualHost *:443>
SSLEngine On
SSLProxyEngine On
ServerAdmin name@nameofyourdomain.com
ServerName nameofyourdomain.com
ServerAlias www.nameofyourdomain.com
ProxyPreserveHost on
ProxyPass / https://192.168.1.37:443/
ProxyPassReverse / https://192.168.37:443/
</VirtualHost>FULL Sample Configuration Reference (DO NOT COPY THIS ONE)
This is what my config looked like when everything was done and working.
The ‘Rewrite engine’ stuff here was added by Lets Encrypt when it was run so it ‘should’ appear in your config after you run it after initial settings have been added. Same with the ‘Include’ stuff and the SSL certificate stuff at the bottom of the second entry.
<VirtualHost *:80>
ServerAdmin name@nameofyourdomain.com
ServerName nameofyourdomain.com
ServerAlias www.nameofyourdomain.com
ProxyPreserveHost on
ProxyPass / http://192.168.1.37:80/
ProxyPassReverse / http://192.168.1.37:80/
RewriteEngine on
RewriteCond %{SERVER_NAME} =nameofyourdomain.com [OR]
RewriteCond %{SERVER_NAME} =www.nameofyourdomain.com
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>
#Listen 443
<VirtualHost *:443>
SSLEngine On
SSLProxyEngine On
ServerAdmin name@nameofyourdomain.com
ServerName nameofyourdomain.com
ServerAlias www.nameofyourdomain.com
ProxyPreserveHost on
ProxyPass / https://192.168.1.37:443/
ProxyPassReverse / https://192.168.37:443/
Include /etc/letsencrypt/options-ssl-apache.conf
SSLCertificateFile /etc/letsencrypt/live/nameofyourdomain.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/nameofyourdomain.com/privkey.pem
</VirtualHost>Random Keywords and messy spam from the Journey
This next section is merely a copy/paste of all the steps I was trying to try to get this working. The purpose is not to follow any of these instructions but merely to leave as keywords in hopes that other people trying the same things will end up finding this blog and save themself the pain! 🙂 So, don’t use the next section for any form of tutorial but feel free to read and learn.
- set up individual virtual host conf files on box 1 else:
We were unable to find a vhost with a ServerName or Address of mydomain.ca.
Which virtual host would you like to choose?
1: nextcloud.conf | mydomain.hopto.org | HTTPS | Enabled
2: ncp.conf | | HTTPS | Enabled
3: 000-default.conf | | | Enabled
Select the appropriate number [1-3] then [enter] (press ‘c’ to cancel):
Select the appropriate number [1-3] then [enter] (press ‘c’ to cancel): c
No vhost exists with servername or alias of mydomain.ca. No vhost was selected. Please specify ServerName or ServerAlias in the Apache config.
No vhost selected
hmm.
finding apache config…
seems like one shouldn’t mess with this… and that lets encxrypt probably does it for you
- sudo apt-get install python-certbot-apache (apparently not installed on ncp somehow..)
- created basic conf file in /sites-available
- restarted apache – worked
- added symlink to sites-enabled, restarted apache, breaks
- run certbot without enabled…with usual
sudo certbot –apache -d example.com -d www.example.com
pi@nextcloudpi:/etc/apache2 $ sudo certbot –apache -d mydomain.ca -d www.mydomain.ca
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
Cert not yet due for renewal
You have an existing certificate that has exactly the same domains or certificate name you requested and isn’t close to expiry.
(ref: /etc/letsencrypt/renewal/mydomain.ca.conf)
What would you like to do?
1: Attempt to reinstall this existing certificate
2: Renew & replace the cert (limit ~5 per 7 days)
choosing option 2
fail. same error above
now trying to go back to simply 443 config in 000-default but wtihout ssl engine stuff.
now running:
sudo certbot --apache -d mydomain.ca -d www.mydomain.ca
this is something… progress….
the bad part:
Failed redirect for mydomain.ca
Unable to set enhancement redirect for mydomain.ca
Unable to find corresponding HTTP vhost; Unable to create one as intended addresses conflict; Current configuration does not support automated redirection
the good part
IMPORTANT NOTES:
- We were unable to set up enhancement redirect for your server,
however, we successfully installed your certificate. - Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/mydomain.ca/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/mydomain.ca/privkey.pem
Your cert will expire on 2019-09-14. To obtain a new or tweaked
version of this certificate in the future, simply run certbot again
with the “certonly” option. To non-interactively renew all of
your certificates, run “certbot renew”
SETTING UP EMAIL WITH YUNOHOST AND CLOUDFLARE
In a previous blog post I set up a Yunohost (“YH” moving forward) box with a script so that it would report it’s location back to Cloudflare (“CF” moving forward) automatically using a cron job entry on the box and a cool piece of free software called ddns-cloudflare. That blog was to make sure the website stuff (ie. WordPress blog, Nextcloud, etc) would work. The other neat part about setting up your YH box this way, I was thinking during the process, is that (I guess but haven’t tested yet), you could just unplug it and plug it in at another physical location (with the right ports open at that location, of course) and it should just start ‘magically working’. This would be a real selling feature for getting ‘off the grid’.
Now to attack the part that most people like me avoid – EMAIL!
We have all heard that email servers are complicated and stressful, but, with the CF-YH combo – once I figured it out – it now seems much easier than I had expected. But there weren’t any specific blogs out there for me to follow so I decided it would be super helpful to write one to help others avoid what I just went through.
This tutorial will connect CF to your YH email and give you a few tips to test as you go until it’s all working, since there are a few things in both CF and in YH that are a bit ‘weird’ I discovered. My hope is that this tutorial helps you get setup faster and easier.
This tutorial assumes you already have a CF account setup with the settings from the previous tutorial (www and A record stuff).
KNOWING WHERE YOUR YUNOHOST SETTINGS ARE
You will be able to find the private and unique details for your own Yunohost installation in the following section of your user interface:
Domains / nameofyourdomain.com / DNS Configuration
When you click this it will open up a pane that has all your records from the previous tutorial but also the recommended email settings. If you are like me, none of it will make sense at all.
The parts you are going to need to match up to CF are:
MX, DKIM and DMARC
The way in which you input them into CF is more than half of the battle, and the part where this tutorial should save you about 3 days of messing around.
First, let me give you a link to Cloudflare’s own support page on this topic. This will also give you a list of pretty much any kind of entry you might need in your own setup, if it’s more advanced than this tutorial. It also shows you how to create a records in your CF DNS settings. Here’s the link.
Now that you know how to enter a record in general, let’s enter them.
I’m going to display this like this:
MX RECORD
- WHAT YH SHOWS IN DNS CONFIG PANE: @ 3600 IN MX 10 mylataylor.ca
- HOW TO ENTER AND PASTE IT INTO CF
- TYPE: MX
- NAME: nameofyourdomain.com
- VALUE: SERVER: nameofyourdomain.com PRIORITY: 10
- TTL: AUTOMATIC
DKIM RECORD
- WHAT YH SHOWS IN DNS CONFIG PANE: mail._domainkey 3600 IN TXT “v=DKIM1; h=sha256; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA…super_duper_long_long_thing”
- HOW TO ENTER AND PASTE IT INTO CF
- TYPE: TXT
- NAME: mail._domainkey
- VALUE: v=DKIM1; h=sha256; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA…super_duper_long_long_thing (NO quotations)
- TTL: AUTOMATIC
DMARC RECORD
- WHAT YH SHOWS IN DNS CONFIG PANE: _dmarc 3600 IN TXT “v=DMARC1; p=none”
- HOW TO ENTER AND PASTE IT INTO CF
- TYPE: TXT
- NAME: _dmarc
- VALUE: v=DMARC1; p=none
- TTL: AUTOMATIC
It was explained to me that I also need to check RDNS, but I have not had any problems yet, and I’m not sure what this is nor how to do it. If you want to add this instruction in the comments that would be great.
If you refresh your page in CF and notice that an orange cloud has re-appeared from grey status, you may not have updated your .yml zone file correctly from previous tutorial. Your script might be updating the DNS records and accidentally forcing it back on. And this will stop your emails from working (the orange cloud). Go back to that tutorial and review the script yml config file setup and make sure you got he hashtags on the right lines…
At this point, it’s the moment of truth: will you be able to send and receive emails?
CREATE ACCOUNTS (AND EMAIL ACCOUNTS) IN YUNOHOST
This part created some issues for me because there may (or may not) be either a bug or an interface issue in the YH account setup. It appears, as you create the YH user that the email can be separate, however, from my experience, you should keep the username and the ’email name’ to be exactly the same – let YH auto-fill it and keep that as your email name. So, if you want your email to be johndoe@nameofyourdomain.com, make your YH username as johndoe at the top and let that auto-fill into the email field below. It seems like YH can’t take periods/dots in the username so john.doe won’t work. There appears to be email aliases that are supposed to work so probably you can figure this out but for me, for this tutorial, I would just avoid dots/periods, keep a simple username and make sure it auto-fills into the email field.
TEST YOUR SETUP
Once your username / email is set up in YH, now move on to test the email, in the client of your choice, but I strongly recommend Thunderbird to at least test to make sure things are working because it definitely works, I can confirm. Once this test is confirmed and you can send, receive emails with a basic thunderbird setup, then can feel confidence about all your settings above.
THUNDERBIRD SETUP
Literally, just follow this link exactly. If your settings are right, it will work. If they aren’t, they won’t. Also, be sure to wait at least 30 minutes for your cron jobs (if you are continuing from the Cloudflare tutorial) to run because the cron job can mess up your settings as well, I discovered. Once you have run a cycle or two of cron jobs and all is well, go wild with the hottest new technology – email…
TIP: when you get to the manual config, Thunderbird puts a period / dot before the serverhostname which is easy to miss. If your email is in the main domain root, then make sure to remove these dots.
Now send a test email to another account you have access to. One important thing to check is that you aren’t ending up in spam folders…
TIP: If you press send on your test email and it hangs on sending, go into your account settings of Thunderbird and you might have some old Outoing (SMTP) servers from previous YH email tests in there. I found after I deleted these old test servers and tried again, it worked perfectly, but if there were other ones in there it hung and failed.
Assuming you got your test email, now send a reply back to it and make sure you get that too. If you’re excited and bored or both, do this step over and over again perhaps with nursery rhyme verses…but make sure no one is watching you… I can’t emphasize enough that you need to wait at least a cycle or two of your cron jobs running to make sure it’s not messing things up over at Cloudflare.
All good? Nice. Another consideration now that you are a warrior hosting your own email is that by using Thunderbird you can back up your emails easily enough by simply connecting and synching your emails across multiple devices.
UBUNTU TOUCH DEKKO SETUP
Now let’s set this up on our Ubuntu Touch device with Dekko.
- Select the left hamburger menu
- Select the top right settings cogwheel on dark panel
- Select ‘mail’
- Select ‘accounts’
- Select the top right + plus sign
- Select IMAP
- Enter ‘whatever you want’ for the first two name options
- IMAP hostname: overwrite example with your yunhost server email location
- ENCRYPTION: should already default to this: ‘force encryption (SSL/TLS)’. if not, do it.
- username/password: auto-filled from first step
- authentication: change to ‘login’ (defaults to ‘plain’)
- SMTP server: overwrite example with your yunohost server email location
- SMTP port: 587
- ENCRYPTION: ‘use encryption (STARTTLS)
- AUTHENTICATION: change to ‘login’ (it defaults to ‘plain’)
Last and final and very important or your outbound email will literally break for this account and, I think, all your email accounts. You need to go back into the settings for this new email address and to this step:
- Select top left hamburger menu
- Select top right settings cogwheel on dark panel
- Select ‘mail’
- Select ‘accounts’
- Select Your newly-created yunohost email account
- Select Outgoing Server
- Scroll down under the LOGIN field and turn on the switch that says ‘Authenticate from server capabilities’
- Press left arrow at top of screen to save settings
CONCLUSION
Now, you should be in business sending and receiving emails from a Yunohost server, in your house, using Dekko email client on your Ubuntu Touch device.
HOW TO SET UP YUNOHOST WITH YOUR OWN DOMAIN (USING CLOUDFLARE)
EDIT 19/06/12 – made some tweaks to this after realizing a few small errors. Sorry if you followed before June 12 🙁
I really wanted to self-host a kind of ‘family box’ which would allow me to have self-hosted email, Nextcloud, websites, and a few other basic things and not have it running on someone else’s server. During the process of searching I came across Yunohost (Pronounced “Why You No Host?”). I installed it on an old test box (super old) with their own documentation and it was really quite simple, especially if you have done any kind of operating system installation before.
My specific goal was to make it all work with a domain that I own (nameofyourdomain.com for this tutorial). I feel that having a strange email address (the default Yunohost email setup looks weird and is awkward) is of little value for most people so this step must be overcome to become a viable solution for myself and other people I know.
OPTION 1 – DIRECT WITH REGISTRAR (EASIEST)
If your registrar gives you full control of your DNS, CNAME, MX etc settings you might not even need this Cloudflare-Yunohost tutorial. My registrar didn’t allow me to do what I needed so I went to the next step. I don’t have enough experience to speak about the different registrars and their settings so research that yourself if you want. Otherwise, move on to this exciting Cloudflare-Yunohost setup…
OPTION 2 – WITH A CLOUDFLARE SCRIPT
As mentioned above, my registrar didn’t make it clear how to do CNAME stuff and mess with MX records, etc, so I ended up searching high and low for an open, free and reliable solution. Thanks to the free software community, I was pointed towards Cloudflare. People I trust and like consider Cloudflare to be ‘good guys’ and that was enough for me to trust and try.
Forgive my limited understanding and description, but I’ll do my best here: Cloudflare is a super robust ‘web traffic controller’ which gives the website admin person (since you are installing Yunohost that is you now!) really powerful control over how data moves to/from the domain/servers. They have cool controls and a nice interface too. Anyway, they have a free account you can start which allows you to do everything in this tutorial and through the process you’ll get a chance to see how nice Cloudflare (“CF” moving forward) is too.
Note: this tutorial assumes you are using Ubuntu or at least have the same terminal commands.
PRE-FLIGHT BULLET POINTS
We’re going to do this:
- Tell your domain registrar to point traffic to CF
- Tell your Yunohost (“YH” moving forward) box to point to CF
- Stick a free software script on your YH box that automatically tells CF where your YH box is every 30 minutes (in case your IP address changes)(replaces dynamic dns service need…)
THE FLIGHT
- Do the YH setup as per yunohost. Make sure your router’s ports are open! Check this page about ports and note that some ISPs will (unethically?) block you from using port 25 (email) and in this case you might be completely out of luck or have to change your ISP so you should check that first on this page. If port 25 is blocked you should be able to use everything except email (nextcloud, wordpress, etc should work) so it’s not completely without hope… TIP! If you get your domain setup first in YH sometimes Cloudflare will be able to magically import all your stuff automatically helping you avoid the manual inputs over at CF
- Get a Cloudflare account
- Log into your domain registrar and change nameservers to the ones shown in your CF account. This guy’s video is pretty good if you haven’t done it before.
- Take note of this project, which is the script which will automate the DNS updates stuff. Special thanks to the programmer!
- ssh into your yunohost box by typing (where 123 stuff is the local IP address of your YH box):
ssh admin@123.123.123.12
This will get you into your YH box where you can stick the script files into your home directory. - Clone the cloudflare-ddns project files above into your YH box by typing this into your terminal (TIP! do NOT use ‘sudo’ here!):
git clone https://github.com/adrienbrignon/cloudflare-ddns.git - Then change to your new directory:
cd cloudflare-ddns - Then change to the zones directory within:
cd zones - Then copy the example yml file so that it duplicates and is named to your own domain:
cp example.com.yml nameofyourdomain.com.yml - Now open the file so you can edit the contents:
sudo nano nameofyourdomain.com.yml - Now edit the ‘admin@example.com’ line and change to the email you registered your CF account with
- Change whatever it says to the right of cp_api_key: to your cloudflare api key. There is a link of how to find that right in the terminal window but in case it stresses you out and you miss it here is the link
- Change zone name (cf_zone:) to: nameofyourdomain.com
- Set all the DNS stuff so that the file looks like this where the # signs are ‘comments’ telling the script to forget about this part:
Only write the subdomain (‘ddns’ for ‘ddns.example.com’)
cf_records:
– ‘@’:
type: A
# proxied: true
log: ERROR
- ‘www’:
type: A
# – ‘ddns’:
# type: AAAA
# ttl: 300
# proxied: false
# log: INFO
If you compare to the example file you can see the changes.
I just commented-out with hashtags the AAAA stuff since apparently I don’t need it (a great contributer told me) as well as the smallest but most painful one, the ‘Proxied: true” line! This one, if you don’t put a hashtag in front, will, every time your cron job runs, tell CF to make CF the controller of the DNS and then basically shut down your websites and your email will also stop working. Then you have to go in and turn the orange cloud back to grey again.
- Save and exit the nano editor with control x and ‘yes’
- Then move back up one level in the directory so you can run next command:
cd .. - Run a one-off test to see if it’s working as per the usage docs with this command
python cloudflare-ddns.py -z nameofyourdomain.com
if it’s ‘working’ you should:
a) see a success report back from your terminal that looks something like this:
2019-05-31 05:16:15,165 | INFO | The record 'www.nameofyoudomain.ca' (A) is already up to date
and
b) should be able to now go to your CF account and see the IP address of where your YUNO-box is / public-facing router listed in the DNS area. You can check this part by going into your router (or one of those ‘what’s my IP address?” websites) and compare your router’s IP address with the IP address in Cloudflare – they should be the same now. If not, assure that the orange cloud in CF is turned off by clicking. It will change to a grey colour when off and this is what you want.
If everything is looking good, let’s move on to making this update process happen automated in the backgroun since it would not be fun to have to run this test script every day or a few times a day!
Now that your config file for this script is all good, let’s go and do the cron job thing
- Type:
crontab -e
Probably if you haven’t done this already it will say it’s blank and give you two options. choose ‘nano’ because it’s easier (option 1) and not VIM because it’s brutal and hurts
- The script provided in the usage page for doing this next step assumes you know what you are doing, that you understand Linux file structures and paths and even cron jobs. I didn’t. So, I’m going to spare you the pain here (you can read the pain below in the bonus section(s) if you are bored or like learning) and tweak this script so that you have a higher chance of this working. First, this is what was provided from the usage page:
Every 30 minutes, update my Cloudflare records.
*/30 * * * * python /path/to/cloudflare-ddns.py -z example.com
If you just simply ran this tutorial that means your Cloudflare directory that you git-cloned in step 6 above is in your home directory. However, you need to add in the user into the path for this to work properly. Also, until you know this thing is working, I would advise you add in the MAILTO option above the script so that you can get a few emails for a few hours confirming it is or isn’t working. You can go back in and remove or comment out with a # the MAILTO line (or comment it out with a hashtag) after you are sure everything is working.
So, here is what I did that finally made it work
Every 30 minutes, update my Cloudflare records.
MAILTO=myemail@myreliablemail.ca
*/30 * * * * python /home/admin/cloudflare-ddns/cloudflare-ddns.py -z nameofyourdomain.com
If everything is working, you’ll keep getting ‘success’ emails that look like this everytime the cron job runs:
2019-05-31 05:30:05,942 | INFO | The record ‘www.mylataylor.ca’ (A) is already up to date
At this point I went back in (see step 17 above) in and hashtagged out the MAILTO= line so the emails stopped coming every thirty minutes.
ADJUST YOUR CLOUDFLARE SETTINGS!
- CRYPTO/SSL SETTING
This one took me an addition day to figure out. I was getting continual TOO_MANY_REDIRECTS errors when trying to access my Yunobox. The problem was solved by the click of one box in my Cloudflare settings as follows:
crypto / SSL : change to ‘FULL’ in the dropdown.
- TURN YOUR ORANGE CLOUDS TO GREY
If you don’t do this step, your email and a few other things won’t work. Just click the orange clounds in your DNS section so they turn grey. That’s it.
DO YOUR LETS ENCRYPT SSL CERTIFICATE ON YOUR YUNHOST BOX
If you try to do your letsencrypt SSL cert before these steps are done, it won’t let you (from my experience) But at this point it should all work. In your YH admin interface, just go to domains, nameofyourdomain.com, SSL certificate, and then ‘install lets encrypt’
CONCLUSIONS
Now your Yunobox should be automatically reporting back your router’s IP address to Cloudflare and Cloudflare is routing your website traffic through its nameservers, etc. As long as your ports and certificates are working, you should now be able to start using it with your own domain.
Now that this is done and you go to your new domain and nothing is there that’s because… there is nothing there. Go figure. So you have to install an ‘app’ (ie wordpress, nextcloud) through the Yunohost app area of admin. I’ll do a separate blog on that probably, but it’s pretty easy.
How to Set up a Calendar with Ubuntu Touch and Nextcloud
1. Set up online account in Ubuntu Touch
You will need all your calendar info from your Nextcloud instance before beginning. Looks something like this: https://yourdomain.com/nextcloudserverlocation/remote.php/dav/calendars/username
- open calendar
- middle icon that shows grid calendar
- ‘add online calendar’
- choose NC
- enter credentials
2. Sync
UT calendar ‘should’ start syncing right away and you’ll see a little icon appear showing that sync started and stopped. Sometimes, however, it seems like a manual sync is required. You can do this by going to the calendar page and selecting the top right hamburger menu and then the circular arrow ‘refresh’ icon.
Recent Comments