Chris X Edwards


Half Power

2016-02-01 09:28

San Diego’s weather is usually famously nice but when we get serious storms the weather can turn quite harsh. Last night we were getting the kind of weather one would expect in the Falkland Islands, basically 50mph wind all night long. This storm caused the most unusual power outage I’ve ever experienced. The lights in the room flickered and some of them went out.

At that moment, my son and I were using four computers (hey, I’m a computer guy) that were plugged into wall outlets. Two of these died. I was able to turn on the monitor of the work station I had been using and I was able to give it a proper shutdown. The last computer was my main server. Because the router and cable modem were also still running, I was able to log into the server with my "phone".

This was so weird I had to understand what was going on. I first tried my Kill-A-Watt meter and got nothing. I tried an electrician’s outlet tester and it showed the outlets to not be live. Of course I checked the circuit breaker and none were tripped open. Since I have a pretty splendid view I could look out over my neighborhood and see that something was very strange. Most lights were off but some were on.

With my multimeter I checked the voltage from the wall. It was about half of what it should be, between 50 to 60VAC. This is not at all correct!

outage halfpower

First is a photo I took outside my lab’s window. The other images show the outage and the low voltage condition. The dark photo is dark because that light is burning very dimly and all other lights are out; my server’s blue LED is still on.

Computer engineers worry about power outages and power surges. The interesting thing to consider from this is should you worry about your equipment’s ability to also operate with a severe under voltage? See this information on "brownouts" to learn more about potential risks.

The other thing to note is that even if an outlet tester shows that the circuit is not live it may, in fact, be somewhat live. Half a dose of normal American electricity probably won’t kill you but it may be a nasty surprise!

Terrible Usernames

2016-01-29 07:58

Many computer users have heard of lists of bad passwords that have been pulled from hacked web sites. Today I was watching tcpdump as an SSH attack was underway and I wondered if there are some particularly bad usernames for accounts.

To find out I scanned through the /var/log/secure file for "Invalid user" entries. The following is a list of all entries I found which were tried 6 or more times.

  • 6 - alex, amanda, angel, app, applmgr, dasusr1, david, db2admin, db2inst1, debian, jordan, joshua, lenovo, leo, lucas, monitoring, prueba, r00t, temp, vps, webadmin

  • 7 - adam, cmsftp, jira, john, matt, student, tom

  • 8 - ADMIN, andrew, dms, jack, odoo, support

  • 9 - a, austin, info, pi, ts3, www-data

  • 10 - backup, demo, dspace, jenkins,

  • 11 - charles, guest, www, zabbix

  • 12 - cisco, richard, ubnt, ubuntu, vnc

  • 15 - tomcat, user

  • 18 - deploy, ftpuser

  • 19 - account

  • 20 - nagios

  • 23 - git

  • 26 - hadoop

  • 29 - postgres

  • 36 - test

  • 37 - oracle

  • 59 - admin

  • 110 - ucsd

My SSH server is not set to "PermitRootLogin" or I’m sure "root" would have been quite popular, perhaps even the most popular.

There were some weird things like 10 instances of this.


I learned many things from this exercise but three stand out. First, be very careful about using certain software products (Oracle, Postgres, Hadoop, Git, Nagios, Tomcat, Cisco); they may be targeted more frequently and perhaps there’s a reason for that. Second avoid generic things like admin, test, account, user, backup, student, temp. Finally, never use your domain name for an account name (e.g. This machine had a address.

If you want to play this game yourself and you have a Linux SSH server open to the world, this command will make the list.

grep "Invalid user " /var/log/secure | awk '{print $8}' | sort | uniq -c | sort -n

Weak Password?

2015-12-25 04:55

I often have to set up Unix accounts for people. People who will comfortably live their entire lives having no idea what an SSH key is. Getting the password to these people can be a trick if they’re not near me. I often leave the password written on a piece of paper somewhere they can pick it up at their leisure. The problem I kept having was that good passwords always have homograph problems. That, combined with my very bad handwriting, makes using a randomly generated password nearly impossible.

This is the method I came up with which seems to cure that specific aspect of the problem.

echo $( tr -dc 'abdefhqrADEFHLQR2347@#$%&*=?' < /dev/urandom | head -c10 )

In theory this will produce (somewhat) random passwords which can be hand written with very little ambiguity. They’re also obnoxious enough that users tend to change them right away. I’ll let the entropy philosophers debate whether this is a "good" password or not.

Simulated Car Racing 2015

2015-12-10 14:06

I’ve been meaning to write a post about the 2015 Simulated Car Racing Championships but I’m still a bit burned out on the topic. The reason is that I am this year’s bronze medalist. Yea for me. Although that earned me 100 Australian dollars I did sink a lot of time and energy into this project. It had its ups and downs. It was definitely a keen learning experience and a formidable challenge. After placing in the middle of the pack in 2013 I had already started to assiduously prepare for 2014… which was the year they stopped having the competition. Grr. As if to mess with me, they started it up again for Summer 2015 under different management. Scrambling to get my entry ready in time, I gave it a go.

Basically this competition involves writing software to control a simulated race car and then racing it. Even though the car and its track environment are simulated, I think the contest would be more correctly called the Simulated Car Driver Racing Championships. In 2013, I created a very nice framework for people who wanted to enter this competition using Python. This was called SnakeOil. This approach does a good job of getting a car around the track and providing a great starting point to dream up fancy driving mechanics. This is great if you like Python, but don’t be discouraged with C or other languages (even Java) if that’s what you prefer. I chose Python because it was what I could easily use to come up with a lot of new ideas quickly with minimal fuss, especially when it comes to the inordinate amount of text processing required to handle the server messages. However, this may have been a wrong approach. Here is my important conclusion: I now believe almost all success in this sport comes from optimization. Perhaps this isn’t any surprise to the computer science researchers who participate in AI (optimization) conferences which feature these contests, but it was a learning moment for me to realize this the hard way.

My bot was, I believe, the most complex of the bunch. This sounds overly grand, but here’s the thing - it doesn’t much matter and can even be counter-productive to have such a baroque system. For example, I spent a lot of time perfecting realistic clutch action on my bot; on the other hand, Autopia, the perennial grand champion (which aspirants should study closely), never touched the clutch. When I realized this I thought, how can this be? This brilliant winner is essentially stupid! The answer is that in what little it did implement, it didn’t make any goofy mistakes and its creator optimized the hell out of all coefficients. Actually, that was my second year strategy. Autopia only has a dozen or so optimized coefficients. I had 57.

This brings us to the real work of the project. You have your bot and you’ve carefully separated out all of the tweakable values into a "parameter file" or some such package. Now you need to divine optimal parameters. Like my ancestors, I used a process of mutation and subsequent mating of the "genomes" of my most successful specimens (simplified, top performer got 100 chances to mate, 2nd got 99, etc). My big innovation to genetic algorithms was encoding not just the value of the parameter in question, but also encoding in the genome the order of magnitude by which it was safe to mutate this value. This, in theory, allowed values that should only be adjusted very finely to encode that information in with the genetic parameters themselves. While I am sort of timidly proposing that this may be an effective way to implement genetic algorithms, I’m not sure my project constituted a serious enough test of that conjecture. It was not a complete failure and it still seems like a neat idea. For all we know, biological genetics works that way, although I am definitely not seriously conjecturing that.

I downloaded the entire corpus of about six thousand tracks, which was quite tricky. Don’t ask how that trade secret works. In any optimization problem you can’t help but train "for the exam" to some extent. This means maximal diversity in tracks is good. This year’s competition found my car at a loss when races were run on classes of tracks I’d never heard of and, even after requesting more information, still know nothing about. Not much to be done about that I’m afraid.

Once you have diverse track environments and a way to mutate your bot’s genome, you must simulate typical life cycles to elicit a representative level of fitness. Here’s where the creative Python approach starts to break down. Python allows you to quickly and creatively add very fancy (i.e. human inspired) logic in a mostly reliable bug-free way. Python can easily keep up with the performance demands at a human scale. But to effectively breed winners you must condense time so that your army of possible contenders are racing billions of test kms orders of magnitude faster than humans can observe them. Suddenly Python finds itself in danger of being the weak link in the chain. It can be done, but I now think it is ultimately a slightly suboptimal choice.

I may have addressed this problem more seriously, but just getting this whole simulation environment to run smoothly in a non-interactive way is a huge tedious distraction. (My notes on such things.) My advice is to study the other entries. If you really want to win, just implement Autopia with your own optimized parameters. And good luck because Autopia did a damn good job of it. If you just want to play around with a bot that drives very strategically and uses a lot of fancy interesting human inspired logic my SnakeOil is a good platform to explore. Also keep an eye on curious things like Ahoora. In 2013, this bot was panned as last place and near worthless as a competitor. For this reason I completely overlooked it. Then one day wanting to see what an "ordinary" entry looked like against mine, I set up a race and was amazed to see last place Ahoora crush not just my humble entry but Autopia as well. Crush. I tried some more tracks and it was completely and universally dominant. Confused, I then set out to recreate the 2013 competition and observe for myself this enigmatic entry. Only then did I discover that it could not drive with noisy sensors, one of the two required regimes of the contest. While noisy sensors were a minor inconvenience to Autopia and most other bots, including mine, for some reason Ahoora was nearly completely paralyzed by unreliable inputs. The lesson is clear that the competitions feature more than a bit of pure luck and many great ideas and much good work is missed by the competition format. An example is having the bot figure out the general coefficient of friction of the track as it proceeds and then dynamically adopting a parameter set bred for those conditions. I think Ahoora did something like that and in hindsight its the best idea I did not think to implement.

I found the actual competition to often be a bit arbitrary and frustrating. Not knowing what type of tracks to expect was definitely a severe limitation to non-luck-based approaches. What the competition did do was provide a structured deadline for getting something working reasonably well. The greatest pleasure I took from this experience was when I would test the best of a new generation of cars against the best competitors and the race was close. Mostly my cars didn’t beat the best but sometimes they did. But when they were perfectly matched, it was quite a thrilling spectator sport to watch the battle and think to myself, wow, I made your mind.

Microsoft Linux?

2015-12-02 14:46

Check out this official Microsoft blog post "Announcing availability of Debian GNU/Linux as an endorsed distribution in Azure Marketplace". Basically this means that Azure is proudly offering to set you up with a Linux virtual machine in their Azure service. Although the post strains to point out that this is a Debian (+GNU, yw RMS) distribution, the fact is that it will be customized enough by Microsoft for this purpose that it could thought of as its own distribution (like Xubuntu to Ubuntu, for example). A Microsoft Linux distribution! I remember when it was Microsoft’s customers who were between a rock and a hard place regarding OS selection.

Azure is even generously offering a $200 free credit to try this superior operating system out.

I would definitely give it a spin if I wasn’t still recovering from third degree cloud-burn. Wondering if Azure has the same fatal problem as AWS I started to look into it. I even called Microsoft because they said to call with questions.

The guy I talked to honestly and quickly admitted that my question was interesting and that he didn’t know. He took my email address and promised to email me further resources; he never did.

See update below.

But perusing the free trial FAQ I found this telling question.

What happens if I exceed the $200 free trial credit?

If you exceed your $200 free credit, we will suspend your free trial account. You can optionally upgrade your trial to be a Pay-As-You-Go Azure subscription at this point if you want to continue using and paying for services. If not, don’t worry - you won’t be billed anything.

Definitely an improvement over AWS, however, the question that I need answered is "What does a Chinese bitcoin pirate need to do to upgrade my trial account to be a Pay-As-You-Go account with near unlimited capacity?" Because as Amazon teaches us, if it’s simply subvert WWW "security", well, consider it done!

At least Azure is full HTTPS. Could be worse.

Update: I did eventually hear back from Microsoft. Which was nice. They didn’t quite completely get the nature of my concern though. I was directed to this page about default spending limits which is exactly my issue. It says

"You will receive notifications once you hit the spending limit for your offer."

This still doesn’t exactly answer the question but it makes it a good guess that if a bad guy takes your free account and "upgrades" it, you will get an email alerting you to that fact. That’s really all I need to feel ok about this. If you’ve simultaneously compromised my web cloud log in credentials and my email (not difficult for many, I understand) then there really is no defense. But AWS didn’t even send me an email congratulating/thanking me for suddenly becoming one of their best customers. That’s stupid. And while Microsoft won’t explicitly claim to be doing better, it does seem like they are. Point to Azure.


For older posts and RSS feed see the blog archives.
Chris X Edwards © 1999-2015