Chris X Edwards

"The media could not be played." Au contraire!
2017-10-18 16:53
Leaf blower, fire alarm, car alarm, and sirens -- all before 0730. So how does one ventilate a soundproof box?
2017-10-18 07:39
I eat very little meat but I did think of a great name for a butcher shop: The Best Of The Wurst.
2017-10-16 08:56
When I first encountered anti-virus software I knew immediately that if I ever wrote a virus it would feature "anti-virus" in its name.
2017-10-13 11:32
The word "handicapped" is completely inapposite to describe deaf people who are talking about lawn care equipment.
2017-10-13 10:34

My Stupid Breakaway

2017-10-15 20:21

In my previous post I shamelessly bragged about my recent graduation from the superb Udacity Advanced Self-Driving Car Engineer Nanodegree Program. That answered the question, what has been keeping me so busy this year? Now I will address part two, why. Why would someone do this and specifically why would I?

Let’s start with an obvious standard question. Would I recommend this educational experience to others? My answer is perhaps surprising—not necessarily. Make no mistake, obviously Sebastian Thrun knows probabilistic robots. Course director David Silver is a super cool guy and a terrific guide to this challenging material. All the instructors were excellent. Everyone I interacted with, both actively and passively, earned my respect. The course material was extremely solid and well organized. I felt that the Udacity platform and support were all well in line with the tuition expense.

Furthermore, and this is important, I feel like if I had to hire engineers for my autonomous vehicle project I would heavily favor graduates of this program. I know for certain that I did learn a huge amount about the topic. This was absolutely a very effective use of my time and resources.

So what could possibly be problematic? Well, on the SDCarND homepage Sebastian Thrun (president of Udacity, famous AV pioneer, lecturer in the course) says this.

There’s an enormous market for self-driving car engineers. Lots and lots of companies that you wouldn’t suspect have entered that field and are massively hiring.

When I first started the program this kind of credulous promotion seemed harmless enough. It can’t be denied that in the last year there has been a bubbly rush to join the autonomous car fad. GM and Ford have each spent a billion dollars on autonomous dreams. There are a lot of start ups getting tons of money. And we can plainly see from the news that Tesla and Uber and Waymo/Google all are involved. And no matter what the level of investment today, intelligent people clearly see that the technology exists to make our current transportation modes obsolete; the only question is how exactly will that happen and when? Given all of this, it may seem reasonable to assume "massive hiring".

But there’s one area where I am far more proficient than Sebastian Thrun and that is simply being a normal guy. I excel at not being a celebrated professor of robotics from one of the world’s top technical universities. I don’t like to brag, but I’m even pretty good at not having a PhD. So when Professor Thrun says there is "massive" hiring, I feel like he means that everywhere he goes, everyone he talks to says the same thing, "We just can’t hire enough qualified people." And that’s probably what Thrun hears, but what is actually meant is "We’re having trouble hiring an army of Sebastian Thruns for a business unit that generates no income."

One of the wonderfully valuable aspects of this course was interacting with other students in the forums. I want to highlight this observation someone posted.

Does anyone want to share their experiences from the hiring event the other night? I had a great time there, but for me it wasn’t all positive. For example, the recruiter from Lyft basically cut me off and said "We are looking for people with industry experience." And then I tried to go in depth about some of my projects from the nanodegree, including the Bosch path planning challenge, and she cut me off again and said something like "Unfortunately the Udacity path planning material is not enough for us."

It seemed to me that strong programming skills, specifically in C++, were more important than the concepts covered in this course. The person that I talked to at Uber told me that they pretty much only hire people that have at least a masters in computer science.

I feel this is representative of what I heard from many people (in my final project team, for example) and what I even experienced myself with two major players. This shows a couple of dissonant ideas in the autonomous vehicle business. The first is looking for people with "industry experience". Obtaining exactly that was definitely a minor reason I embarked on this course. And I feel I got it as good as it gets. But the more glaring point is this: there is no industry experience! This industry does not exist! Obviously decades in the computer business show that a couple of years after a fad technology arises, all businesses will be found trying to hire anodyne salarymen with "5 years experience". So par for the course there.

But the second point this shows is that precisely because this has not evolved into a positive-cash-flow "industry" it does not need "engineers". It needs researchers. It’s little wonder that these companies are gutting top university robotics research labs.

My own personal experience has not been much different. Would you rather have some random PhD who needs a job or some insane guy who has made it his personal crusade/hobby to tirelessly accomplish your company’s goals? What about two of the latter (they’re cheap) for one of the former? Nope, that’s just not a thing in this putative "industry".

I do get the feeling that Udacity has taken great pains to try and facilitate finding a job. They’re doing everything they can I’m sure. I think it spills over from their other offerings where there are (unfortunately) millions of Android and web developer jobs, etc.

According to this article the Udacity Self-Driving Car Engineer Nanodegree program has accepted 10,000 students. Shockingly, 43,000 have applied! I know that at the vanguard of this the quality may be a little higher, but the people I observed and interacted with were extremely high quality! If there’s one thing I would observe about this "industry" it is that there is no shortage of extremely competent people who are extremely interested in doing this work. These are great people, serious people. If they are not getting hired by companies that believe they need self-driving car engineers then those companies are deeply flawed.

During the final term my favorite tier one auto supplier, Bosch, sponsored a contest involving autonomous vehicle path planning. The top 25 participants would win an interview for a "Research Engineer" position (note the self-contradictory job title, is it "engineering" or "research"?). Without going into the bizarre miracle that allowed me to be one of those top 25, I did get to talk to the person who makes hiring decisions for Bosch’s autonomous research group. It turns out that while Bosch is a big company with 390,000 people, their Palo Alto offices have only 200. And unless I misheard, the autonomous vehicle group there has only twenty-five people. And as I’ve pointed out before, Bosch is one of the major players in this "industry". If they can’t easily double or quadruple their tiny AV gang from the 43,000 people who are keen to help them do so, I’m sorry, but something is very wrong with that picture.

Then there is the problem of the potential workforce not being Sebastian Thrun enough. God knows Professor Thrun has done everything anyone can be expected to do to share his expertise! What these foolish companies do not understand is that the people who have just completed this course are not cast offs or rejects from "proper" educational attainment. These are very high quality people. They have shrewdly optimized their educational investments. So much so that I will assert that you won’t find anybody better. The reason is that if you are clearly a cut above the people I saw in the program, you can probably get NSF (and other) grants and basically research whatever strikes your fancy on your own terms. Or, quite realistically, you can start your own business.

Working for a big well-funded company can certainly have advantages and I’d love to work for a group with their own private test city to play in. But that’s not infinitely valuable or even strictly necessary at this stage. There is still a lot of software to be written and that costs exactly nothing to get started with.

Standing on the job seeking sidelines it’s been a good show. I’ve played along with the "career development" stuff Udacity offers, but I didn’t take this course to "get a job" (got one thanks). I took this course because I literally can not wait for this problem to be solved by others. I genuinely wanted to know as much as I can know about how best to reduce the amount of human stupidity involved in driving. If you share that goal, then you will be delighted with Udacity’s course. If you simply need a job, well, I’m not so sure it will work out. (I’m totally open to being corrected here but I’m just calling it like I see it.)

When I look at the flurry of recent interest in autonomous driving I am reminded of bicycle racing. I know most people think it’s a very boring group of guys riding bicycles from point A to point B, but it’s actually one of the most interesting game theoretic sports there is. In cycling, riding quickly is actually the anti-goal. The goal is to do as little work as possible on the way and yet still win at the end. I often feel like the major players in the auto industry are mostly marking each other rather than giving it their all; in fact, that is their correct strategy! Very often some antsy rider (stupidly) can’t hold back his wattage and breaks away from the comfort of the main pack’s slipstream. These can often lead to very long punishing solo ordeals. The pack makes sure that they never let the breakaways get too far ahead and, indeed, most are dramatically but precisely caught just a few kms from the finish. Sometimes, however, the breakaway artist pulls it off, the powerful favorites in the main peloton misjudge his strength (or their own laziness) and they can’t quite catch him in time.

My Udacity experience was just one phase in my solo attack. I do not expect to win. I’ll be disgusted if I do! That is not my goal. But know this — a race with a lot of spirited attacking always finishes faster than one without challenges. And I am in a hurry to see the finish line where the misdeeds of distracted, drunk, and idiot drivers are merely a chilling historical footnote rather than something I rationally fear for my life about every day.

I hope that SDCarND graduates do not wait for a job if it is slow to come. Attack this now. Do what you can to move autonomous vehicle technology forward. A fool like me will surely fail, but 10,000 of us fools will not.

I am mother fucking Jens Voigt

The First Self-Driving Car Engineer Graduates

2017-10-13 23:49

Back in March I wrote the following.

"I don’t like to say too much about my accomplishments before I accomplish them so if I manage to get through this program I’ll have much more to say about it."

True to my word, that time has come. I have completed the third and final section of the Udacity Self-Driving Car Engineer NanoDegree course. I started in mid February with (I’m told) the fifth cohort. I worked quickly, skipped ahead, and have now just finished among the first cohort, the first people to complete this training. Here is a brief overview of each three month term.

Term 1 focused on two major topics that were both very valuable to me, computer vision and machine learning. Those are not just each big topics, they are huge. They do necessarily go together for autonomous car purposes and getting all of the necessary knowledge in place was certainly no easy educational challenge. In fact, the big problem with this whole course is that to learn about computer vision or machine learning usually requires graduate level specialization at a university. To apply this to autonomous vehicles usually requires graduate level specialization at one of a handful of elite technical universities. To get some condensed straight information about how to most practically apply these difficult topics to autonomous vehicles was extremely helpful to me. Here’s an earlier write up of an impressive project from Term 1.

Term 2 focused on what I call "rocket science". This included sensor fusion and all flavors of Kalman filtering. That stuff is just straight up 1960s NASA slide rule rocket science. Udacity founder, Sebastian Thrun, is the author of Probabilistic Robotics so there was quite a bit on probabilistic motion. Basically more rocket science. There was also some control theory which was a lot more practical than my university exposure to it (but somewhat less practical than my machine shop exposure to it). This was also the term where C++ was to be emphasized. In addition to the heavy topics, this could have been an additional stumbling block but it was pretty easy for me. My contra-Java obsession with C++ finally paid me back a tiny bit; maybe one day it will pay me back in real money.

Term 3 seemed to have a slightly different style. It was less about hardcore fundamentals and more about topics that one needs to know about. There was some pretty detailed material showing how one could do path planning, for example. I found that interesting but didn’t feel like that’s how I would do it. It’s not how I did it with my SCR entry notable for its quality racing lines. There were two interesting "elective" topics (only one was required). The first was functional safety and basically the paperwork guff that the automotive industry feels is most helpful in baking safety into the design process. I’m not knocking it. Like the ISO9000 guff that was popular when I was a factory engineer, I am both horrified by this stuff while simultaneously admiring its reasonableness. I went ahead and looked at all of the lessons for this just to get familiar with it (my notes on it, for example) but ultimately I chose to do the other elective project. That project was "semantic segmentation". Learning about this has pretty well convinced me that this scheme will be the heart of my planned self-driving vehicle. It’s pretty amazing. I’ll probably write more about it separately.

Finally, at the end of term 3, there was the grand group project. The theme here was "system integration". This was envisioned to be the showcase attraction of the whole course, actually programming a real self-driving car. I led a group consisting of two guys in Germany, one in RSA, and one up the road in LA. Luckily everyone on my team was high quality and we were able to work well together. I even took the opportunity to become an expert Git user despite my traditional misgivings. A lot of the difficulty in this project involved the mundane ordeal of getting a setup that worked. For example, ROS, the Robot Operating System, was not impressing me with efficient minimalism. Many people were forced to compound all of that by using VMs to run the necessary Linux running the necessary ROS on their Macs. Although I didn’t have that problem, there were strange issues with the Unity game engine simulator which caused all kinds of mischief. Anyway in the end we were able to get our simulated ROS-controlled car to drive around in the simulator and properly recognize (and stop for) stoplights in the camera feed using a machine learning classifier. Yay! You can see videos of the project if you’re interested.

The performance of the real car, however, was less inspiring. Udacity had a lot of early development problems getting the car ready for action. They also had to work out complex scheduling. From our point of view, we barely understood what the car was even supposed to do because its mission in real life bore no resemblance to the virtual simulated mission. This was especially painful to me because my main autonomous vehicle area of interest is simulator calibration (for example and here’s another example). There were also issues involving different libraries available in the simulator VM and on the car. When we got our two minutes of drive time, the car basically did its lap around a small parking lot and ignored the fake stoplight prop. In our defense, they had changed the camera quite a bit from the material we had been given to train with (nope, the dashboard never moved with respect to the camera) and, the whole operation was done in the dark half an hour after sunset. Of course we had no idea that was a possibility and no way to train to spot this particular fake stoplight in the dark. I think the real car testing was an ambitious idea that had some necessary extra difficulties for us as the first students to try it out. I’m confident that they will refine the course and it will continue to improve.

Although this course was advertised as requiring approximately 10 hours per week it was later acknowledged that this was rather optimistic. I spent at least double that. Considering that and the $2400 price tag, this was not a casual learning experience. I actually felt more engaged for this course than I did for my engineering degree from a traditional university. I would even say that this course featured more challenging material. I felt like the lessons were really well designed and presented. For me, on-line education is so superior to traditional live lecture performances that it’s hard for me to imagine how the latter still exist. (And I work for a charismatic university professor!)

I’m very happy with my decision to take this course. I learned things which I wanted to learn and things I didn’t know that I didn’t know, things that are indeed important to my goals. I feel like if I talked to anyone who was building an autonomous vehicle at this time I would never be completely in the dark about what they were talking about. I feel like I understand the state of the art in this field better and I better understand some people’s AI optimism even if I still do not share it. I even feel like I can immediately apply many of the course’s lessons to my day job; machine learning and vision are useful in many fields. Overall I have been delighted with this experience.


Review: Everybody Lies

2017-10-12 18:54

Upon finishing Everybody Lies by Seth Stephens-Davidowitz, I thought that it probably could be a condensed into a long magazine article. And I was right-- here is one by the author himself covering noteworthy highlights so I do not have to. Or if that’s too long, just the subtitle tells the story, "Big Data, New Data, And What The Internet Can Tell Us About Who We Really Are". I’m not saying the work involved to derive these insights was small, but really, you’re interested in the answers, right?

SSD (the author) basically believes that the hugeness of our single-point-of-failure internet giants provides enough statistical muscle to make some new and bold assertions about the world. Basically if a billion people search for something no one thought other people were searching for, that is interesting. I can’t imagine any reasonable person disagreeing with that.

Unfortunately, this new insight tends to show us that this hidden world is generally worse than we have supposed. Indeed, I think that this book contained the most depressing sentence I have ever read in my life: "If there is a genre of porn in which violence is perpetrated against a woman, my analysis of the data shows that it almost always appeals disproportionately to women." If you think I’m being overly sensitive about it consider the magnitudes, "Fully 25 percent of female searches for straight porn emphasize the pain and/or humiliation of the woman…" and "Five percent look for nonconsensual sex…" That’s one in twenty women looking to watch a rape on a popular porn site that actually forbids such content.

After that, the fact that people are horrendous racists almost seems banal. Oh, ya, people are horrendous racists it turns out. By the way. FYI.

The titillating and depressing content of this book is all over the internet. I’ll let you seek it out if you dig that sort of thing. I will simply say that the data is substantial and the methods seem credible. SSD wrote a book. Cool. Moving on.

SSD likes to think (probably correctly) that the insights of this kind of data can shed light on victims who normally do not have a voice, especially when they are being oppressed by more a more powerful group. The classic example is child abuse. This got me thinking about another vulnerable group who are viciously victimized by a powerful group who are good at preserving the secrecy of their horrific misdeeds. That brings me to my job where I spend some of my time trying to penetrate the shroud of secrecy erected by the pharmaceutical industry.

If you’ve been to the doctor in the last 100 years or so, what happened? More than likely he gave you some magic beans to eat. So here’s a funny question, how do you know those magic beans are not poison? Think about it—if a bunch of people who went to a doctor drop dead, well, that happens, right? So how do we know those magic beans are up to snuff (better than snuff!)? Most people will just say "uh because… doctors?" Some very stupid people will say "the free market". Some intelligent people will say, "Ah, the FDA!" Some might say "science" or "trials". But that actually does not tell you if the magic beans you’ve been prescribed are poison. All that can do is give you some hints about how to extrapolate a trial of hundreds (normally beset with corruption) to a blockbuster market of tens of millions. What could go wrong? So surely once 10 million people are taking the magic beans, there is some kind of review of how that’s actually working out. Right? There must be. Right?

Sorry. No.

I’ll save my work on the embarrassing (to the medical profession) FDA Adverse Effect Reporting System for another time. But churning through that stinking landfill of bad data looking for meaningful scraps has put an idea into my head. If SSD’s methods can detect flu outbreaks, etc. they should also be able to reveal what kind of problems people who use Atorvastatin (Lipitor) actually have. This is not unknown. Pfizer knows exactly how lethal this poison is just like Catholic priests know exactly how many boys in the parish are getting sodomized. What’s missing is some sunshine because these guys are really good at keeping it in the dark.

So if you’d like to walk away with some optimism after learning these depressing things, consider that knowing the truth can ultimately inspire positive changes. The techniques of this book do seem able to shine a light of truth, pleasant or not. Let’s hope that light does some good.

Shucking Ridiculous

2017-10-11 08:28

Here in California we have vending machines that will fill up your container with drinking water. Imagine one of those machines from a fictitious supplier, SeaGreat, that would fill up a 5 gallon bottle for one dollar. Imagine this machine is outside of a convenience store that is always open and which sells sealed 5 gallon bottles of SeaGreat water. The question is, how much do the full complete bottles of SeaGreat water cost? Obviously, they must be more than one dollar. Obviously. Imagine going inside the store to see how much money you were saving by bringing your own bottle.

What if it was not more than a dollar? What if the full bottle was $.75?

If that were the case you would have to reconsider one of the most fundamental ideas of economics, that one plus one is greater than one. Well, start reconsidering! Because in the crazy world of computer hard drives, this is exactly the situation. It turns out that at this time it is clearly less expensive to buy an external hard drive than the equivalent internal model.

Here are some examples.



The bigger the drive, the worse it gets.


This shows an extremely rare (perhaps discontinued) internal model but that was as cheap as I could find. I just liked how these came up together on the same search result. Looking at results for "8TB internal" the only drives with more than 1000 reviews were $274, $325, and $307. Also several people in the reviews for the external drive report that it does contain a 7200 RPM drive, so forget the tempting idea that the performance specs must somehow be different.

Just to make sure you’re clear about what’s going on, here is a $23 box that allows you to convert an internal drive into an external one.


You would think that an external drive would cost as much as the sum of this box and an internal drive. But no!

It’s not just I who have noticed this. There’s even a special term for the process of extracting an internal drive from an external enclosure. Searching for the word shucking turns up all kinds of confirmation of this weird situation.

Here’s a good recent article about it. But it’s not even a recent thing. I’ve been sensing some confusion about this for years and it seems that is accurate. This discusion from 2013 is simply titled "Why are External HDDs cheaper than Internal HDDs". No good answers are proffered. And same story in 2015. Here’s another asking the question; no proper answers are even attempted and comments are skeptical that the situation can be possible.

Knowledge is power and my purpose is to alert people to the fact that something very strange is happening in this market. If, like me, you sometimes buy stacks of large internal drives it can be worth exploring options that would seem ridiculous. That said, careful research may be required. Some people believe that recent integrated USB controller boards will make external drives incompatible with SATA controllers.

Hopefully some clever people are using this as an arbitrage opportunity and breaking apart external drives and selling the parts. Just be aware the next time you’re shopping for hard drives and something seems crazy—it’s not you.

Greatest Technological Accomplishment Of The Last 1000 Years

2017-10-04 11:25

Dear Silicon Valley,




(Yes, I meant 20th.) Special Hut ab to my friends at Bosch for making this possible (for me); my Bosch power tools have exceeded all expectations for 14 years.


For older posts and RSS feed see the blog archives.
Chris X Edwards © 1999-2017