Gcubed

Conspiracy Detector Web App

2017-08-13T00:00:00+00:00

Guess what!

I have finally created the definitive killer app, Conspiracy Detector.

Wait, didn’t it already exist?

Fair enough, I am plagiarizing myself a little bit. If you have poked around this site before, you might be aware of my previous write up and portfolio item Conspiracy Detector. In fact, they are both based off of the same code, with this new version being a responsive web app.

So what pressing need does it solve?

Why the need to be able to paste a blob of text into some stranger’s website and have it spit back out a heavily opinionated verdict on whether or not the text was written by someone off of their lithium of course!

Is it a sexy, sexy website?

Well, you are welcome to check for yourself, but in my opinion its rather spartan look is actually a statement against the oppression and commodification of the worker by the great neoliberal machine, a verbalization of the angst of the common man as the Ideological State Apparatus slowly imprisons us all within the confines of our own minds, all the while eating the orange and throwing away the peel in the words of Arthur Miller.

In all seriousness though, the site is not the most beautiful to ever grace the interwebs. I designed it to be as minimalist as possible. It gets the job done and does not waste your time loading nor does it waste the rather limited resources of my mid 2000’s former thinclient server serving.

So how does it work?

The website is powered by flask, a microframework for python websites. Simply put, it takes the input of a text box, puts that string through the exact same code that gives predictions on the terminal (well, minus the tokenizer which slowed my server down too much and also caused some issues with loading my pretrained model), and serves a new page with the verdict.

Hasta la vista, baby!

And on that note, I hope you enjoyed my write up on Conspiracy Detector. If this interests you, go ahead and check out the web app here, the source code to the original here, or even my portfolio section on the project here, not to mention my original and more detailed write up here.

Update I have now just gotten around to posting the source code for the web app on GithHub here.

I can’t shake the feeling that all these conspiracy theorists are in on it. Wynne McLaughlin

Conspiracy Detector

2017-06-25T00:00:00+00:00

What are you babbling about now?

The scientific detection of clinical paranoia of course! This post is intended to complement my portfolio entry here on my conspiracy detector project, which I posted the code for on Github here .

Update I have also put together a web app that you can access here. Unfortunately, because I am ~~self hosting (on the same box as this site)~~ now hosting this site on Heroku (because setting it up for their free tier was a whole lot simpler than trying to run a server accessible to the wider world on a university network from my dorm room, that model has a simplified tokenizer to save processing power and is thus less accurate.

How exactly did this happen?

Over the natural course of reducing false positives with my email spam filter, clearly! In truth, I kid you not, important emails were being filtered as spam and during my quest to fix this deep disturbance in the force, I was completely sidelined by my curiosity in the actual mechanics of spam filters. A few quick searches later and all was revealed to me… The secret sauce is math! Conditional probabilities and a naive Bayesian filter to be exact.I leave the actual specifics of the workings of this model to Wikipedia, but it suffices to say that it is a machine learning model which when given examples of inputs belonging to different categories is trained to draw inferences between certain characteristic of an input and its label, thereby allowing the trained model to make predictions and classify previously unseen inputs such as emails into spam and ham (legitimate).

I then followed the obvious train of thought of using this type of system to determine whether or not Trump was the progenitor of a tweet. This jump was ultimately unsatisfactory, mainly because I was having a hard time finding a corpus of tweets that were not from Trump and training binary classifiers with only examples from one category leads to pretty disappointing results.

Fortunately for my tinkering, I stumbled across this hilarious blog post entitled The Quantum Hitler Scale shortly thereafter, and decided that what the world was missing is a heavily opinionated bot capable of discerning text containing some semblance of rational and coherent thought from the literary works of the brave resistance fighters who refuse to be brought down by the Man and His Lithium pills, and who instead tell it like it is, staring death by shape-shifting, alien lizard monsters from the center of the Earth right in the eyes.

And so, the conspiracy detector was born, a state of the art system powered by pure math (well really, applied math) with a twist of computer science.

So what does it actually do?

What Conspiracy Detector does

Just as its name says, this weekend project (actually developed over summer break over the course of two weekdays) is a system that uses machine learning to detect conspiracy theories. In other words, I applied the miracles of modern mathematics and computer science to the prickly problem of detecting people who are totally off their rocker, or in the words of the verdict that my machine gives, “bonkers!”.

What Conspiracy Detector does not do

An important caveat is that this system does not detect bias or even fake news (a much tougher problem because it can look pretty similar to reputable news sources). Instead it focuses on the previously (at least to the best of my knowledge) untackled problem of statistically determining if a given piece of text is absolutely nuts (shape-shifting, Jewish, alien lizard nuts).

Another important factor is that this system is entirely constructed from my biases. It believes that the New World Order of Dick Cheney and the Communist, gay, Jews is a conspiracy theory (although I would not put it past Cheney), precisely because I think that this is a completely off its rocker conspiracy theory constructed by a lonely and chemically imbalanced talk show host, terrorist, or pseudo-intellectual who forgot to take his or her pills. If you do not agree with my assessment of what constitutes a crazy conspiracy theory, then you will not agree with this systems verdicts which reflect pretty strongly exactly what I think of the likes of Alex Jones, David Duke, Lyndon LaRouche, William Jennings Bryan, and company. In other words, if you believe Conservapedia, then this bot is probably not for you.

How does the magic happen?

Yep, almost indistinguishable from dark magic, this system is actually powered by math (also, python)! It uses a support vector machine (SVM), a machine learning model that can be used as a binary classifier, meaning a much fancier spam filter which is capable of assigning new inputs (after training) to one of two categories (so bonkers and not bonkers in this case).

Give me some technical details!

The code is written in python3 and uses sklearn, an awesome python library for all types of machine learning that I highly encourage you to check out, for the majority of the machine learning heavy lifting. It also uses nltk (a python library for natural language processing) as part of the pipeline in the tokenizer for word stemming.

And what about training?

A large amount of this project, like many machine learning systems, was training data collection. In order to obtain these thousand of plain text samples necessary to train the system, I first quickly put together a web scraper in python to download articles from news sources such as CNN and BBC, as well as from my daily dose of dismay and despair for humanity (Infowars articles). To complement this, I then dove into the bowels of the Gutenberg Project and Archive.org, using old Infowars magazines, as well as “papers” from esteemed intellectuals like “Dr.” David Duke, Lyndon Larouche, William Jennings Bryan, Paul Joseph Watson, and so many more to obtain the irrefutable textual manifestation of clinical paranoia. On the more rational side, I populated the training set with the plain text English translations of several works by Dostoevsky, Tolstoy, Balzac, Flaubert, Hawthorne, and so many more, as well as through several bland scientific papers. I then split these documents into files of 100 lines each and removed all of the bizarre special characters which had infected some of these texts, eventually leaving me with a grand total of over 2000 training files.

A slight problem

When I initially built this system, it was pretty heavily based off of sklearn’s documentation examples. Unfortunately, due to David Duke and his love for Jews and communists, I had to tinker around quit a bit with tokenization, the process by which the words of the input text are split up and prepossessed, in order to prevent my baby from coughing up a lug nut every single time a text had the word “communist” or any variant of the word “Jew” or “Jewish” in it.

So, how did you fix it?

After quite a bit of tinkering, I ended up adding among several minor things like stripping out white spaces and punctuation. This reduced the number of bizarre two letter features that would dominate the model’s conception of a conspiracy theory, but did not solve the underlying problem of the repeated use of certain similar words. To solve that problem, I added word stemming to the tokenizer of the model. This strips all words to their roots, thereby reducing words like “jews” to “jew’ and preventing silly things such as plurals and derivations from causing too many problems. This solved my problem and improved overall accuracy quite a bit, but also slowed down the training process because of the added complexity to tokenization (although it is still very fast, taking only two minutes on my laptop from 2007).

Does it work?

Short answer: Yes!

Longer answer, quite well. On the test cases during training, the model attains 99% accuracy. Moreover, it has passed my ultimate litmus test of distinguishing between a Washington Post article and an article from Conservapedia (a Christian fundamentalist version of Wikipedia that among other things believes that Obama is an Islamist Terrorist, that Liberals are pretty much genocidal maniacs, and that conversion therapy works).

In case you were curious,, it has also determined that my about page is for the most part not crazy, and neither is the Communist Manifesto.

And now some charts and tables

After each training session, the system extracts the top ten features that you can see here:

Top 10 most important features:

Rank	Weight	Bonkers	Weight	Fine
1	-1.91936	o	0.964309	change
2	-1.21264	jew	0.87151	wildlife
3	-1.09732	government	0.82531	reynard
4	-1.09311	ei	0.805876	trump
5	-1.08023	louse	0.805513	senator
6	-1.06831	n	0.773819	specie
7	-1.062	txt	0.753482	â
8	-1.01539	ee	0.749294	prince
9	-0.999346	obama	0.719461	sea
10	-0.930854	written	0.703385	energy
11	-0.91092	jewish	0.678142	place
12	-0.901538	magazine	0.672481	climate
13	-0.886269	medium	0.670092	important
14	-0.885437	american	0.655379	point
15	-0.829171	right	0.651944	section
16	-0.807909	st	0.648213	photon
17	-0.7689	d	0.641958	orofino
18	-0.761795	litical	0.641508	king
19	-0.744603	generator	0.619919	conflict
20	-0.742536	freedom	0.60663	himself

Or, if bar charts are more your speed, you can see the top 20 features for each category here:

Now, some weird things come up, mainly on the crazy side, because the input text is still a little dirty despite several attempts to clean it up. However, the system actually works quite well as you can see from its confusion matrix and classification report (from the test set):

Classification Report:
             precision    recall  f1-score   support

    bonkers       0.99      0.99      0.99       298
       fine       0.99      0.99      0.99       303

avg / total       0.99      0.99      0.99       601


Confusion Matrix:
[[294   4]
 [  4 299]]

Overall, the top 10 features seem to indicate that a lot of conspiracy theories tend to focus on Jews, Obama, and the Government, as well as on our freedom being taken away, as the words “American” and “freedom” seem to indicate.

Farewell

And on that note, I hope you enjoyed my write up Conspiracy Detector. If this interests you, go ahead and check out the source code here, or even my portfolio section on the project here.

I stand by these selections and QHP, but a principle is not enough. No, after reading thousands of articles of alluring bullshit promising me oneness with Earth, good health, eternal life, and infinite persecution by secret kabbalahs bent on world domination and keeping me from knowing the truth about vitamin C, I thought, “Why can’t my computer sort this out for me?” I wanted a program that I could plug in that would tell me if the author of an article was bulldozing bullshit, and I wanted to know how much and how fast. I wanted a Quantum Hitler Scale. Peter Welch

An intro to Prisoner’s Dilemma

2017-03-05T00:00:00+00:00

The first musings

I have recently been reworking a computer science and mathematics workshop that I have been running with the United Innovators , and I thought that I might as well share my thoughts on just how interesting I think it is.

What is Prisoner’s Dilemma?

Normal Prisoner’s Dilemma

Prisoner’s Dilemma is a classical game theory problem in which two prisoners are given the option to defect (betray the other) or cooperate (stay tight lipped). These actions carry different incentives depending on the actions of the opponent that can be represented like so:

Thus, when playing a single game the best option is always to defect because it minimizes the potential cost while maximizing the potential reward, especially since the opponent cannot retaliate later.

Iterated Prisoner’s Dilemma

However, things get more interesting when we spice them up with a twist on the game, iterated prisoner’s dilemma. In this case, the players play multiple successive games, both aware of the other’s past moves. This means that there is a possibility for retaliation, creating a need for more nuanced strategies.

Through the Lens of Game Theory

Game theory is as Wikipedia puts,

The study of mathematical models of conflict and cooperation between intelligent rational decision-makers.” Wikipedia

In other words, it deals with how to maximize rewards and minimize negative outcomes when competing and cooperating with other selfish parties.

If we translate iterative prisoner’s dilemma into the terms of game theory, this means that we get a game that is:

Symmetrical: Both parties have equal power
Non-Zero Sum: Net result is not necessarily 0, so both players can win or lose
Simultaneous: Players make their moves simultaneously
Imperfect Information: Each player does not know what the other one is doing
Discrete Game: Moves are discrete and binary

Some History…

At this point I think that its interesting to inject some historical context into this problem.

Would you like to play a game?

Yes, I did just manage to tie this in with Wargames. Prisoner’s Dilemma actually does originate from the Cold War when the United States was struggling to figure out how to keep the Arms Race under control without over-escalating to war. Around the time of the Cuban Missile Crisis they came to an organization called the Rand Corporation, an American nonprofit global policy think tank made up of mathematicians, economists, and public policy experts. These academics subsequently modeled the scenario using the mathematical model of Prisoner’s Dilemma and made pioneering working studying this problem, all to avoid nuclear war (or win one when they thought it was possible to).

Axelrod’s Tournament

The Tournament

In the 80’s a professor named Robert Axelrod decided to try a new approach to Prisoner’s Dilemma. He setup a round robin tournament in which he excepted code submissions from around the world to compete as bots. The interesting thing is that he discovered that it is the “greedy” strategies, those that are the most selfish, that in the long run did the worst whereas within that scope it was the more altruistic strategies that began to shine. This lead him to conclude that cooperation between organisms might actually be the result of natural selection, with the most strictly selfish (over the short term) behaviors that in the end lead to failure.

So, which strategy won?

Tit for Tat

While it obviously depends on the competitors, the winner of the tournament was a remarkably short and simple strategy called Tit for Tat. The rules for it were simple:

Always start by cooperating
Repeat the same move the opponent did the move before In other words, this algorithm tries to quick start a virtuous cycle of cooperation, then rewards further cooperation from the opponent with cooperation and penalizes defections by retaliating with a defection. This avoids the costly situation of cooperating when the opponent defects, while still attempting to construct a virtuous cycle of mutual cooperation, the only move in the game that truly benefits both players.

Variants of Tit for Tat

Over the years, a couple of variants of Tit for Tat have emerged. Some like Two Tits for Tat are a little stricter when it comes to retaliation by retaliating with a defection for two turns instead of one, while others have introduced the concept of forgiveness, that the bot has a certain probability of cooperating even if the opponent defects.

My own strategy!

The entire concept of Prisoner’s Dilemma, the mix between geopolitics, mathematics, and computer science, fascinated me enough that not only have I run a workshop on it (and I’m planning on doing another one soon), but I actually came up with my own original strategy called Dynamic Two Tits for Tat which you can take a closer look at in my portfolio here . In fact, if you want to take a closer look, it is now a strategy integrated into the Axelrod repository, an awesome open source project that you can take a look at here, and which allows you to run full blown matches and tournaments between strategies (and so much more)!

I digress. My own minor contribution to this field is another variant of Tit for Tat. It is actually based off of Two Tits for Tat, but integrates a probability of forgiveness, meaning cooperating even when the opponent has defected within the last two turns, which is calculated as the ration of the opponents number of defections and total number of moves:

This represents in a sense just how nice the opponent is, and thus allows my bot to determine how forgiving it should be of its opponent based off of this (which is also the odds that any move by the opponent will be a defection).

The Only Way to Win is Not to Play…

Or maybe just to outsmart them with a better strategy (just kidding, nuclear war is unwinnable).

And on that bright note, I hope you enjoyed my write up on the Prisoner’s Dilemma. I can’t encourage you enough to check out the Axelrod repository here, or even my portfolio section on my strategy here if this interests you.

Dr. Strangelove: The whole point of the doomsday machine is lost…if you keep it a secret! Why didn’t you tell the world, eh?! Russian Ambassador: It was to be announced at the Party Congress on Monday. As you know, the Premier loves surprises. Dr. Strangelove: Or How I Learned to Stop Worrying and Love the Bomb

Sexy, sexy custom 404 pages with Apache

2017-02-02T00:00:00+00:00

What is this 404 page you speak of?

So, you just put together a beautiful webpage when lo and behold, you mistype the url and get a terrifying looking message like this:

It’s like the creators of this message were trying to destroy your carefully crafted aesthetic experience, and maybe give your visitor a cold shiver down their spine from this brutally ugly and utterly unhelpful message.

Save me!

Fear not though! This can be fixed with less effort than you might imagine. Unfortunately, most tutorials that I came across seem to be meant for Wordpress sites and/or tell you to use the .htaccess file, something which the folks who built Apache very explicitly tell you not to do on their site with:

You should avoid using .htaccess files completely if you have access to httpd main server config file. Using .htaccess files slows down your Apache http server. Any directive that you can include in a .htaccess file is better set in a Directory block, as it will have the same effect with better performance. The Apache Team

So, how did you manage it?

The good thing is that this is pretty easy to do right using virtualhosts and just a little bit of tinkering (we are running this on Linux after all).

Create your 404 page

First of all, create a beautiful 404 page named 404.html and make sure that it is in the root web directory of your site.

Some inspiration

Now, if you are lacking inspiration for your page, the magical interwebs will most certainly come to your rescue if you just search for custom 404 pages. Alternatively, you can check out mine here .

And onto Apache

This part is actually quite simple. Just open the virtualhost file of your site with: sudo nano /etc/apache2/sites-available/example.com.conf (example.com being the name of your site’s virtualhost file) and add the line: ErrorDocument 404 /404.html under DocumentRoot.

Now, if you used certbot to setup https, you will need to repeat the same process with the /etc/apache2/sites-available/example.com.conf-le-ssl.conf file it created.

Tip If you followed my previous tutorial about setting up your server and building your site with Jekyll, then just modify as needed your theme’s 404 file (probably names 404.md) and then follow along with the virtualhost configuration.

And Voila!

Done! Your visitors will now be graced with a beautiful custom 404 page.

Thank you for dropping by, and

So long, and thanks for all of the fish!

Douglas Adams

How I built this site!

2017-01-05T00:00:00+00:00

Remember, remember the 5th of November?

Or, my previous post. Well, if you don’t, in that post I announced the creation of this site, outlined its purpose, and promised to put together an article on how I put together this website, mainly because it was a bit of a pain to find all of the information I needed to setup and self host Jekyll, and I think it would be helpful to have it all in one place for the sake of my sanity and possibly yours.

Step one, the hardware

While I do have and Raspberry Pi Model B+ lying around (used for among other things a personal IRC server), I decided that because this new webserver would be hosting both this website and also an Owncloud installation (a personal cloud server and possibly the subject of another post), it would need to run on something a little beefier. That is where the Dell OptiPlex FX160 comes into the picture.

The OptiPlex FX160 is an old thinclient server, meaning that it was used by companies to run Citrix installations that their employees could remote into. The best part is that a whole lot of them have flooded places like Ebay for cheap because it is relatively old hardware that has just gone off lease.

There are a couple of nice things about this box that led to me picking one up:

They are cheap (I got mine with 3 GB of RAM and a 240 GB hdd for $55, only marginally more than the cost of a Raspberry Pi)
They are fast:
- Because I can hook up the hdd directly over SATA, the IO bottleneck that made my Pi painfully slow as a webserver is no longer an issue.
- With 3GB of RAM in my case and a 1.2 Ghz Intel Atom, it is pretty snappy.
They are really energy efficient (not much worse than a Pi).
They are dead silent (fanless).
They look pretty sexy in my completely subjective opinion.
And why not?

A couple of pictures of my sexy, sexy home server. And yes, I know I am a master photographer.

And now, onto the software end of things…

If you were like me and thought that the hardware end seemed a little too easy, welcome to the fun part. First stop, Debian.

I chose Debian because it is fast, lean, and very stable with a wide selection of software. Kind of like Ubuntu without all of the bloat they add to Debian. Anyways, Debian owes being lean to people who take the expression

My program is user friendly, it is just very selective about its users

to heart.

A good example of this is quite simply the fact that it is very likely that the last time the Debian webpage was modified was in the mid 90’s (and yes this is the webpage of the same project that runs much of the world’s banking institutions among other things). For instance, it is laughably difficult to actually find the download links for the Debian iso, so to save you a bit of time (to make up for the time you just spent reading my tangent), here is the torrent link and here is the regular over http download link.

Anyways, it is now time to start setting up and configuring your shiny new Debian system:

The OS

The Install

Installing Debian is actually pretty easy. Just

Burn the iso to your USB (using dd or unetbootin)
Plug the USB into your soon to be server
Turn on the future server and hit the function key that lets you select the media to boot off of
Select your USB and hit enter
Select install Debian from the GRUB menu
Follow the instructions
Sit back, relax, and enjoy the ride

Basic OS setup

Update and install

First of all, let’s make sure all of our software is up to date: Log in with root credentials (we have not set up the normal user with sudo privileges yet) and run:

apt-get update && sudo apt-get upgrade && sudo apt-get dist-upgrade

Then reboot for good mesure.

Now what we are going to do is setup ssh so that we can run the server headless:

apt-get install ssh openssh-server

Make sure that your favorite text editor is installed:

apt-get install nano vi

Setup your user

Now, lets make things more convenient by granting your normal user sudo privileges:

Install sudo:apt-get install sudo
Grant privileges: usermod -aG sudo username
Open your /etc/sudoers file with visudo or nano /etc/sudoers
Make sure the line under # Allow members of group sudo to execute any command looks exacly like this: %sudo ALL=(ALL:ALL) ALL
Restart the SSHD server with: /etc/init.d/sshd restart
Logout and then back in with you new non root user (run logout)

Tip You use ctrl-w with nano to search the file and ctrl-x to close it (hit enter when it asks about the filename and y to confirm).

Setup ssh

Now, lets make things a little more secure by forcing key based authentication for ssh and disabling direct root login:

First disable root logins by opening /etc/ssh/sshd_config and changing the line PermitRootLogin to PermitRootLogin no (you might need to get rid of the # prefixing it)
Create a key by running ssh-keygen -t rsa on the computer you will login in to the server with (make sure to back these keys up)
Now run scp ~/.ssh/id_rsa.pub username@ipofserver:.ssh/authorized_keys (I’m assuming you’re running Linux) on the same computer to copy the public key to your server
Test by restarting the sshdserver with /etc/init.d/sshd restart and then trying to login to your server again (on Linux by running on your local machine ssh username@ipofserver)
If that worked, you can now disable password logins by opening /etc/ssh/sshd_config and changing the line PasswordAuthentication to PasswordAuthentication no (you might need to get rid of the # prefixing it)

Warning! If you are considering exposing this server through ssh to the outside world, you should look into something like port knocking and authentication rate limiters like fail2ban.

Setup a firewall

Another easy thing you should do to harden your server is setup a basic firewall. I chose ufw a wrapper for iptables, because it is a lot easier to quickly setup and for something like this we don’t need that much configuration

Install ufw with sudo apt-get install ufw
Make sure it is configured to handle IPv6 too with sudo nano /etc/default/ufw and making sure that IPv6 is set to yes with IPV6=yes
Lets put together some basic rules: run sudo ufw default deny incoming and sudo ufw default allow outgoing to block all incoming connections and allow all outgoing connections from the server
Now, lets make sure that we don’t lock ourselves out when the firewall is brought up by allowing ssh (port 22) connections by running sudo ufw allow ssh
For good measure, enable http (port 80) connections and https (port 443) connections with: sudo ufw allow http and sudo ufw allow https. That way, we won’t be blocking any web connections when the web server is up and running
We are now ready to start the firewall with sudo ufw enable
Check that all of the rules are setup with sudo ufw status. The output should resemble this:

Status: active

To                         Action      From
--                         ------      ----
22                         ALLOW       Anywhere
443                        ALLOW       Anywhere
80                         ALLOW       Anywhere
22                         ALLOW       Anywhere (v6)
443                        ALLOW       Anywhere (v6)
80                         ALLOW       Anywhere (v6)

Automatic security updates

Another good thing to do in order to harden our system, after-all we are going to be exposing it to the internet, is to setup automatic security updates.

First, we need to install: sudo apt-get install unattended-upgrades apt-listchanges. If you get an error about the package not existing, add the following to the end of /etc/apt/sources.list

# backports
deb http://ftp.debian.org/debian jessie-backports main

Next, open /etc/apt/apt.conf.d/50unattended-upgrades and add the line Unattended-Upgrade::Mail "root"; inside the Unattended-Upgrade::Origins-Pattern block
Now, just run dpkg-reconfigure -plow unattended-upgrades in order to configure unattended-upgrades (just hit yes when it asks if it should automatically install certain updates
And we are done!

So now the server is a little more hardened and we are ready to install the software for our web-server:

The software

In order to get this web-server up and running, there are a couple of pieces of software that we will need to install and confgure. These will be apache for the actual web-server, a client to update the IP for the server’s domain name, and certbot for our free SSL certificate.

Port forwarding

Firstly, we are going to need to setup port forwarding. This will allow you to expose your server on specified ports to the outside world (which is a little necessary for a web-server).

Depending on your router, you will probably need to go to 10.0.0.1, 192.168.1.1, or 192.168.0.1 with your browser. Enter your username and password (if you don’t know it, it is probably either written on your router or the default one which you can find online: in which case change it!).

Now, find the menu for port forwarding (probably under advanced) and set it up to port forward ports 80 and 443 (http and https) to your server’s local IP address which you can get by running sudo ifconfig.

Getting a free domain name

Once that is out of the way, you should now get a domain name. This allows people to type in the domain (for instance www.google.com) and have it redirect to your server’s public IP address which can change and is harder to remember. While you can pay for a domain name, it is just as easy to get a free, albeit slightly uglier looking one.

Several companies such as NoIp provide free domain names, so you can just go over to one of their websites and follow their instructions to register one and set up a client on your server.

The web-server

We have finally reached the point where the humble computer gestates into a beautiful web-server. There are a couple of different ways to do this, withapache and nginx being the two big solutions. While both work, and nginx might be a little faster in the context of serving static web pages, in my opinion apache is waaaaayyyyy easier and quicker to setup, especially if later on you want to setup something like Owncloud (a personal cloud server) later on.

Installing Apache

First of all, we clearly need to install apache, which is as easy as running sudo apt-get install apache2.

You can test that this worked by going to ipofyourserver with your web-browser.

Configuring Apache Virtualhosts

The next and last thing that we will need to directly configure with apache is virtuahosts. These allow you to run multiple websites with different domain names from the same server.

First, we need to create a root direcetoty for your new web-page. Run sudo mkdir -p /var/www/example.com/public_html with example.com being the domain name that you picked up earlier.

Create a demo page for this site by copying the default demo page for apache wiith

sudo cp /var/www/public_html/index.html /var/www/example.com/public_html/index.html

Next up, we need to changethe permissions of this new directotry structure so that apache will be able to serve our web-page. Run
```
sudo chgrp -R www-data /var/www/example.com && sudo chown-R www-data /var/www/example.com && sudo chmod -R 755 /var/www
```

At this point, we need to create the virtualhost file by first copying the default one with

sudo cp /etc/apache2/sites-available/000-default.conf /etc/apache2/sites-available/example.com.conf

Customize this file by opening it with sudo nano /etc/apache2/sites-available/example.com.conf and then
1. Setting ServerAdmin admin@example.com to your email
2. Setting ServerName to example.com (your domain name)
3. Changing DocumentRoot to your root directory which we set earlier to DocumentRoot /var/www/example.com/public_html
Enable this new virtualhost and site with sudo a2ensite example.com.conf
Restart apache with sudo service apache2 restart
Test this configuration by navigating to your domain name from outside your local network (for example using your phone connected with data)

Tip! You can use virtualhosts to setup and run multiple websites with different domain names from the same box. Just repeat the same steps for virtualhosts as above, but change the domain name and root directory. You will probably also need to reconfigure your dynamic ddns client to point your new domain name towards the same box, and also rerun the certbot step (up next) to have both sites use the same certificate.

A word of advice! If you want access this site from its domain name from inside your network, you will need to modify your /etc/hosts file if you are running Linux so that that domain resolves to the local IP of your server.

Setting up https for free!

At this point, the web-server is setup and serving a basic page when we connect to it. It even works with a domain name! However, we might want for the sake of security to enable https, and thanks to the wonderful people from Let’s Encrypt , it is not only ridiculously easy, but also free.

They even have a great guide here that shows you how to obtain that nice, shiny green lock.

Finally, the actual website

At last, you are nearing the end of the road. You are just spitting distance away from the random and wise quote, but alas, you have not yet achieved your goal of a beautiful website. You put your faith in me, and I abandon you at this junction with just a pitiful single html page to show for it. But fear not! I have not forsaken you!

In this part of the tutorial I am going to give you a quick overview of Jekyll , the system I used to build this site. I will then show you how to self host, using the web-server we just setup, the content you create using Jekyll.

A quick intro to Jekyll

What is Jekyll?

So, what is Jekyll? In the words of its creators, (or at least the creators of its website),

Jekyll is a simple, blog-aware, static site generator perfect for personal, project, or organization sites.

How does it work?

Basically, the way it works is you take a preexisting theme (or make your own) that defines a bunch of layouts (reformatted pages) and includes (snippets of code to include in pages). You then write your content mainly in markdown and specify which layouts and included to use. Jekyll then compiles your page into html and cssthat the server will serve.

Themes, did you say themes?

The great thing aout Jekyll is that it is pretty widely used, so there is no shortage of free, open source, and sexy themes. If you google jekyll themes, you’ll get a bunch of pages of themes like this one and this one with massive lists of themes. Check a few out, and then just follow the instructions on its webpage to use the theme of your dreams. Jekyll and most themes have pretty good documentation, so between the two of them, you should be pretty well set for building your site.

For reference, the theme I used for this site is called Minimal Mistakes.

Some words of wisdom! In case the documentation does not make it clear, you do all of your development of your webpage on your computer and only push the final compiled result to the server.

Great, but how do I host this?

So, you just put together a sexy, sexy site, the most beautiful to ever grace the interwebs, but wait, how do you actually put this on the interwebs? You may have noticed that the bulk of the explanations for hosting your Jekyll page involve using Github Pages, but you’re here because you want to do this yourself.

Github, we don’t need no stinkin Github! A famous movie or something

So how do you self host?

It is actually pretty simple. You just have to push the contents of _site to your root directory of your web server. The easiest way to do this is to do all of your development on your local machine, and then open your terminal inside the root directory of your Jekyll site (where you can see all of the files and directories starting with _). Then, just run the following to copy the contents of _site onto your server:

 jekyll build --destination _site && scp -r ./_site/* username@ipofserver:site

If that gave you a directory does not exist error, run mkdir ~/site on your server first.

After that, move these files to the root directory of your website (get rid of the demo file first) with:

sudo cp -rv site/* /var/www/example.com/public_html/

Now, fix the file permissions with:

sudo chgrp -R www-data /var/www/example.com/public_html
sudo chown -R www-data /var/www/example.com/public_html

And presto, you are done!

Some words of wisdom! Whenever you want to do this again (say, when you update your site), make sure you delete the contents of ~/site on your server before you rerun all of these steps starting with scp.

Some more advice to spare you the hair pulling! If after transferring your built site to the server you get weird looking pages and a bunch of 404 messages, make sure that you only copied the contents of _site into your root web directory and not the directory itself.

And so, it is now time to part ways. Good luck on your journey to build your own website, and I hope this post helped you maintain a little but of sanity.

I’m not quite sure what my next post will be right now, but I’m thinking some game theory is coming up…

Until then, good luck and good day to you!

The more you find out about the world, the more opportunities there are to laugh at it.

Bill Nye the Science Guy

Guess what: A brand new website!

2016-12-05T00:00:00+00:00

Guess What?

I finally got around to building myself a website! Repeat, I finally got around to building myself a website.

Why Should You Care?

Honestly, you shouldn’t. It almost certainly will not have any life changing effect on your life (if it does, please let me know as that may qualify as a miracle and make me eligible for sainthood).

The True Meaning of it all…

42, no wait… The true intention of the website is fourfold:

Actually put all of the eclectic devops and webdev skills I have acquired for various reasons to use for me.
Provide a centralized place for me to put any random thoughts I may have, as well as any tips or tricks I have come across in my projects that I would rather not forget and that might be helpful for you, my dear reader.
Host a portfolio of the projects I have worked on.
Because it just seemed like fun to put together and self host a website.

Back to the Future!

Soon enough this website will have more then enough content to satiate the appetite of anyone, even those willing to read the musings of some random internet person, so please stick around and check back soon.

Spoilers

Currently, I am planning on putting together my first article/tutorial on what else but the creation of this very website (hint: the day Debian works out of the box is the day that chickens will have teeth).

Bye!

Until then, have a nice life, but first a random and wise quote from your favorite always smoking Frenchman.

Man is the only creature that refuses to be what he is.

Albert Camus