I think I’m finally fully recovered. Whew. Last Saturday afternoon I
got an email from the power company saying, oh hey, we’re turning the
power off tonight, sorry about that. Such a thing is always a
possibility but still never welcome. Since I’m pretty serious about
avoiding interruptions of computer service, it is something I try have
a plan for. But on this occasion I had a little problem.
First let’s introduce my main personal server, raven.
Raven lives on my desk and is on 24/7 where it does all of my
important things. Email, cvsroot, most programming, blog stuff, help
notes, and general writing all happen on this server. It doesn’t have
a monitor or a keyboard. Instead, I log into it over SSH from wherever
I happen to be. It is my own personal cloud.
One of the important features of this machine is a
pair which means that everything that is ever written to disk is
actually written to two disks simultaneously. In the event a disk
fails, the other carries on with no loss of service. I’ve been using
this setup since 2005 and have written
about it before.
This machine also runs a Xen VM hypervisor and the real host of
interest is a Xen VM. This means that if something causes problems
with the VM, it’s easy to restore a backup image of it and try again.
And when I say "something" I mean Gentoo update. Oh ya, it’s all
Gentoo, of course, because I’m insane.
Anyway, many months ago, I updated the hypervisor and I watched with
some apprehension as GRUB got switched out for GRUB2. Normally, that
kind of thing in the computer world is progress, but GRUB2 is a huge
pain in the ass and plagued with challenges. So much so, that it is to
me, basically unusable. Thanks to this updated bootloader, I wrote the
following note to myself.
There was some emerge shenanigans recently and GRUB seems to have
been changed from legacy to GRUB2. I don’t know why GRUB2 wasn’t used
to begin with but it looks like I attempted it and then gave up. Maybe
I’ll find out why.
I created a new /boot/grub/grub.cfg (none of that makefile bullshit)
and I ran grub-install which I think is pointing to GRUB2 these
days. Hard to tell really.
I anticipate there will be reboot problems. Raven is up and fine now,
but if a reboot happens, it may be very hard to restart because the
hypervisor won’t want to boot. This must be tested.
All screens on raven will have to be shut down. After the attempted
reboot fails, a monitor and perhaps a keyboard will have to be
arranged for xedxen and troubleshooting can begin. Chances are good
that sysresc will be needed.
And guess what? After 10 hours of no power I discovered everything in
that note came true. Since you don’t care about this nearly as much as
I do, you’re asking, what’s the point here? Who cares?
This situation got me thinking about my server design and I came up
with some modifications.
There is no escape from GRUB2. It simply must be used. This is
problematic because, ironically, doing complicated things with it is
much harder. The reason for that is that they worked very hard to
make it much easier to do complicated things but they completely
failed. Now instead of a well-documented configuration file that can
be configured, there are distro specific automagical configuration
systems that "compile" into the configuration file which, if done
this way, is not human readable. My solution - use GRUB2 in the
simplest way possible.
I want RAID1 and GRUB2 maybe can do this, maybe not. Don’t worry
about it. Don’t let GRUB2 even try. Make GRUB2 boot the simplest
thing possible and let that system figure out anything more
I like to have my kernel do what is needed because it is set up
exactly the way it should be. And in 2005, this worked for something
like mounting a simple Linux system that just happened to be protected
by a kernel managed software RAID1 double drive setup. But as time
wore on, the metadata for the RAID volumes changed and although the
new bootloader understood it, the kernel, weirdly, did not. No one
seemed to care that these formerly robust systems were now broken.
This was because *every*body used an initramfs so that the kernel
wouldn’t have to do its job. I fought this and tried to do it the
Right Way, but eventually I just ran into too many brick walls.
But with the simplest boot setup possible, one does not need an
initramfs. And this is good. I’m also weird in that I do not load
modules. Any of them. If I need kernel functionality, I compile it
in. Normal off-the-rack clothes are not better than bespoke tailored
ones just becuase the former are popular.
Magnifying the complexity considerably is the Xen hypervisor. Since
I felt I needed this setup to be a VM, that’s what needed to happen.
But it turns out that this is fine. In fact it’s a perfectly
reasonable answer to all of the above problems.
The solution to all the problems is to take the preboot complexity of
GRUB and initramfs and the right kernel modules and a RAID1 setup and
exchange it for a Xen hypervisor (which I was already committed to
anyway). The idea is that instead of running all these very complex
things (GRUB2, initramfs) to provide for what will be needed on the
complicated running server, just boot a hypervisor in the simplest way
possible. Then assemble your RAID1 disk set and select the right
kernel with the right stuff and fire up the VM.
The only change from my previous system is that the hypervisor is not
also on the RAID1 system. (And now finally the point!) Here’s the
thing, it doesn’t matter! If I do everything with my VM, and nothing
with my hypervisor, why would I ever need RAID1 with that? In fact,
it’s better to not have that be mirrored in real time.
On my hardware (you can kind of see it), I have 2 small micro SSDs and
2 normal SSDs. The former were the RAID set for the hypervisor and the
latter were for my real server. But I realized, why not just set up
one of the micro SSDs to boot the hypervisor just fine and then copy
that setup to the other one. It wouldn’t be live, but if I messed up
the live hypervisor (as I did with the GRUB clobbering update), I
could actually just boot up the reserve one. Since no value gets added
to the hypervisor drive once its all up and running, there’s no reason
for RAID1. And that cures all of the problems.
Now once the hypervisor is happily running with absolutely no
complicating elements, I can put together the important RAID1. Then I
just use that raid set for the important server. Here’s my Xen
configuration showing the multi-disk device md127 being mapped to
the VM’s /dev/sda1.
disk = ['phy:/dev/md127,sda1,w'] # <-- raven
Let’s not ask why it’s 127 and not 0. I’m not happy about it, but I’ve
learned to just accept that madness.
I’m not normally a huge fan of
fiction. Conceptually it is fine, but realistic spaceship stuff can
be pretty sterile. The problem with spaceships is that they’re just
not nearly as interesting as they were in the 1960s and 1970s. Back
then it was clear from the exponential growth in space travel that
by 2017 we’d all regularly travel to space for holidays. As a kid I
personally had absolutely no doubt that my profession would be
astronaut. But the funny thing about exponential progress in an area
of technology is that it doesn’t necessarily go on like that forever
(ahem AI enthusiasts).
Far worse than spaceships for me are stories with zombies. Zombies are
a stupid trope that has been quite overused in the
last decade or
so. And if you tell me a story has vampires, I flat out won’t even
look at it. Vampires are just too camp. I can only tolerate
mockery. If you told me a book has spaceships, vampires, and
zombies, well, no, that’s just of no interest to me.
And yet I really liked Echopraxia by Peter
Watts which indeed had all of the above. The key to the book’s
success with me, I think, was that it was modern. It’s harder to
maintain enthusiasm for books by smart people like
Asimov saying 1960s stuff
that was no doubt fascinating then, but less compelling today. The
author claims in the end notes (n.b. there are end notes) that he
really tried to reach for something different and I think he fully
I’m still working on my fiction grading criteria, but I’ll go through
what I’ve got so far.
Style - A lot of panache. Quite complex but not soporific. Clean
grammar. Solid readable prose for someone with a PhD.
Organization - Good. No 80 page chapters. Creative and functional
Character names - Ok. Mostly not irritating or incomprehensible.
Believable characters - Good. No problems. Even with advanced,
modified, and alien intellects.
Natural dialog - Worked. Seemed believable, even for advanced
intelligence characters which can be very tricky.
Plot complexity - Excellent. Uses some hard philosophy to make some
really tricky plot devices. Cerebral but superb.
Plot resolution - Excellent. The bar is low here. Like a Tolkien
book, this is an out and back story. That’s good enough, but it’s
more subtle than that too. Some of it I don’t think I really even
understood through no fault of the author.
Erudition - World class. Amazing. A huge depth and breadth.
Gems - Fair. Watts' entire style is so energized that punctuating it
with special flair may not be practical.
Immersion/world building - Very good. It was all a little flashy,
but there was a lot of it. It is hard science fiction because there
was attention to detail and plausibility.
Believable tech - Good. It was pretty outlandish, but all vaguely
plausible. I’m sure Watts would say specifically plausible, hence
the serious end notes.
Imagination/creativity - Superb. Bringing the aforementioned
vampires and zombies out of the stupor of their tropes was
masterful. If anything, I would almost say it was too imaginative.
I first checked out this author by reading
some of the work he has
publicly available on his web site. I was impressed enough to read
more. He sounds like a cool
guy too. "He spent ten years getting a bunch of degrees in the
ecophysiology of marine mammals and another ten trying make a living
on those qualifications without becoming a whore for special-interest
The other approach, my approach, is to figure out what we could have
been doing ten years ago, and just get on with it. Today! To better
explain the general concept I’m trying to outline, I’ve thought of a
specific example of my strategy. What follows is a strategic outline
that could, from a technical and practical standpoint, begin to be
Let’s start with the company that has done the most to advance
the state of the art of autonomous cars. No, not them; I’m talking
about Tesla. Tesla is in a unique position to
really make a big difference. They’re not a perfect fit for my
strategy, but for illustration purposes they’ll do.
Tesla already makes cars. They don’t exactly make the right kind of
cars, but in some important ways they do. The fact that their cars are
important. The fact that they have cameras and enough sense to be able
to do their lane keeping mode is helpful. But, as we know, the best
hands free driving in a Tesla still demands that the driver pay
scrupulous attention. This is the thing that I think is critical to
eliminate as quickly as possible where possible.
So we have a Tesla that darn near drives itself, but not really. What
progress can be made? Here’s the strategy that Tesla should pursue.
Find popular long travel routes. A classic is Los Angeles to Las
Vegas. Or Los Angeles to San Francisco.
Prioritize these routes by how monotonous they are to drive. Also
prioritize routes that see no snow and little rain. The stretch
of I-8 between Yuma and Gila Bend is a perfect example.
Start collecting Tesla Autopilot
data on these routes. Note any quirks or problems that ever come up.
Add site-specific software if need be. The goal at this stage should
be to show that a significant number of cars made this trip with
zero human input. Maybe more than would be expected to do so without
a human mishap.
Build some special lots (think truck stops) right off the exits at
each end of the long stretch. For example, one just east of Yuma and
one just west of Gila Bend. For all intents and purposes, there is
nothing in between but hours of very plain driving. These lots would
be run by Tesla and they would have several functions. The primary
function would be that a Tesla in this lot should know how to get on
the freeway and then drive to the next lot’s exit, get off the
freeway there, and park in that lot. That’s it!
Rent out Teslas. In this scenario, I should be able to rent a Tesla
in San Diego or LA and drive to Yuma the normal unsafe irritating
way. Maybe I could use attended autopilot where feasible, but that’s
not too important. When I got to Yuma, I would pull off to the
special lot, verify my plan and see if the road conditions are ok
for such a thing, and turn over complete unsupervised control to the
car for the next couple of hours. When I get to the Gila Bend lot, I
wake up and resume driving.
When I get to my destination, say Tucson, I’ll have a car to do any
flexible driving I want, but for this round trip, I’ll knock hours
off of it where my attention and energy are required.
One tricky paradox with this is that Teslas, being electric, are not
ideal cars for crossing the desert, an environment which can barely
supply fuel to gasoline powered cars. The answer is that these
special lots should also do battery swaps. This is especially true
for the rented cars.
For some reason this vision of how autonomous cars could have been
working for our safety and sanity a decade ago is repugnant to fancy
AI specialists. They seem to feel that 1% of ideal is a glass 99%
empty and would prefer just keeping that glass 100% empty until they
can make it 100% full. Which should happen about the time that
computers will make transportation obsolete for other, hard to imagine
I believe that by establishing a foothold of that 1%, it will inspire
2%. And then 4%. Soon almost any monotonous long drive should have an
option to swap out your battery and have the car take you down the
boring road. A lot more people would be buying Teslas. If Tesla was
smart and used open standards for the batteries and certifying the
road safe for simple autonomous use, then other companies could make
compatible cars and people would demand the system be expanded with an
exponential network effect. I believe that very soon we would be at
the 95% we are at today with research cars trying to handle any road
like a human. The difference will be that there will be a lot more
real use of real autonomous vehicles. That final 5% will be easier to
cure not just from that experience, but because it’s the final 5%; the
will to just get rid of it and replace it with an easy environment
will be much greater.
What I’ve described is just an example of working with the AI that is
perfectly good today and not perennially wishing and hoping for some
science fiction AI to get here in 5 to 10 years. I think the
Otto truck transportation concept is closer to mine
than many others. Perhaps with my kind of system, problems like rain
and snow can be avoided by simply not allowing cars to be fully
autonomous in such conditions. Sure, that’s not the best situation
imaginable but it might be the best situation that is actually
useful today. It would certainly highlight where more work truly
should be done. Today the work seems focused on the impractical quirky
situations that vex even human drivers, situations computers are
probably never going to properly deal with without a lot of special
Not only am I proposing that this kind of plan can happen. I’ll go
farther and make another prediction: that it will happen. In rough
terms the way I’m describing the advent of autonomous cars will
happen and current well-known efforts like Google’s will fail.
Autonomous cars will first begin to be useful by the public only in
some limited and incremental form and only on certified roads.
The question I have is really, when? Should we start today? Or
should we wait for another 50 years of AI research to try to bring us
closer to the human-level intelligence needed to drive like a human?
A goal that may be unattainable, especially if the last 50 years of AI
research is anything to go by.
Here’s an article titled
left Twitter. It is unusable for anyone but trolls, robots and
dictators. I haven’t left Twitter because I never really went
there. I’ve had an account since 2008
(apparently) but I never really used it. There are some old Tweets
there because I did hook it up to my own messaging system that I
developed until the (proprietary) delivery service failed. But I’ve
never logged into Twitter and actively used it as intended. Every once
in a while I take a look to see if I’m missing anything, and I never
get the feeling that I am. Here are some of the reasons I feel that
One of the promises of Twitter is that it can, in theory, provide very
immediate information about what’s going on in very specific places.
But I’ve never had that work out. For example, my balcony has a nice
view of a large urban area and I once saw a massive fire raging below
which produced smoke that blackened the sky over much of the city. I
wondered if anyone on Twitter nearer to it could provide more
information about what it was. Nope. No one I could find even
mentioned it. Maybe it was just my Twitter ineptitude, but I was
surprised and definitely hoping for better.
Ok, one should not expect highly local timely news. What about other
"news" you can get? Problem one is that spurious news is a big topic
today. Could Twitter be part of the fake news revolution?
so. I liked how
article entitled Facebook, Twitter and Google bosses to be grilled
by Parliament over spread of fake news had the compulsory but
nonetheless ironic "Follow us: Facebook/Twitter" at the bottom.
The "news" on Twitter is not local, timely, helpful, or even true.
What’s else can be found? For me one of the most irritating things
about Twitter is the juxtaposed spaghetti graph of bewildering
"conversations" with "replies" and stuff. Imagine if you were reading
this text and I took a moment to say—Oh and Bob, let’s definitely
do that thing we talked about!--You’d be thinking WTF? But then if I
just kept doing it—Your arguments are completely stupid, Mr.
Squeaky—you’d really want—If you don’t believe me, Tom, check out
Russian dashcam videos!--to stop reading. It’s very much like when
someone interrupts an in-person conversation to rudely take a cell phone
call. I’ve been known to just walk away from that situation.
Let’s assume you can filter that stuff out (though probably only
for each conversation).
I would be looking forward to a delightful stream of small messages
micro-blogged for micro-attention spans. But that’s not what Twitter
is, is it? The 140 character message concept is a fiction. Why do I
say that? For a while now Twitter has been mailing me six tweets every
day. At first I looked at them (in Mutt, my text
email client) but soon realized that they would never be interesting.
I went ahead and collected a few weeks of these and analyzed their
Twitter probably is taking special care to send me only tweets that
are meaningless without a marked visit to Twitter (which I
scrupulously avoid) but that’s not terribly encouraging. Here’s an
idea of what typical tweets with links look like. (I’m including the
mystery links for completeness, but I sure don’t recommend clicking on
And where do the links really take you? Perhaps terriblemalware.ru.
Who really knows?
Although such links don’t help without further investigation, retweets, can be
even more bewildering. This one, for example, doesn’t seem to include a
specific link but I assume that it is implied by the very concept of retweet.
The ten albums that defined my teenage years were probably all s3ms.
I have no idea what that means but at least it’s the right format.
The fact is that Twitter is just as much about referring people to
other things as it is about publishing any content and it is not at
all rare that this other material is more than 140 bytes. Sometimes it
is weird prose stories, quite long, broken into awkward 140 character
packets. Really, Twitter is a baroque URL shortener.
This seems to be a fundamental collapse of the internet. It seems that
to a certain extent web search engines have been replaced by celebrity
tweets for guiding people to information. Perhaps this is excusable,
sort of like Digg or StumbleUpon but with more celebrity influence,
but I think it’s really messing with some key principles that made the
internet compelling in the first place. For example, it’s definitely
shameful how many "140 character tweets" are nothing more than links
to non-searchable images containing vast amounts of text. That’s
just the worst of all worlds.
Of course if you love celebrity news, and I know that’s by definition
really popular, Twitter is pretty great. I just noticed, for example,
that Elon Musk does a fairly good job of
making his own news so reading his tweets would be like being plugged
into what would have been press releases in the old days. But most
people aren’t all that newsworthy.
So what of it? Well, I have mixed feelings about it all. If people
want to have a way to say short simple things and if those short
simple things are essentially links to noteworthy resources, fine.
But Twitter, Inc. kind of creeps me out. As I mentioned I had
developed my own system, a Bash script, for generating and archiving
my short messages and I liked it very much. I used a service called
ping.fm to replicate these
messages on Facebook and Twitter. Here’s an important fact,
however—that company died before I did. Big deal, you might say, but
the bigger important point is that if you’re believing there is no
chance that Twitter itself can die, you’re not looking at a long
enough time frame. Just look at the pitiful state of Yahoo to know
that big companies do not equal a safe haven for your data. This is
especially true when the true product is you. For example, here is
cool hombre JWZ reminding us of this from the comments
of a Perl script he wrote for
archiving Twitter messages.
# Unfortunately, Twitter seems not to let you retrieve anything more than
# about a year old. So for those old messages, you're just screwed.
# That's what you get for trusting your data to someone else!
Recently I came across my old status message program and I enjoyed
re-reading the safely archived content.
Since my grand criteria for whether I publish something is whether I
think I’ll enjoy revisiting it, I decided I should revive that
tradition. Since archiving is important to me, I won’t even get into
the mess that JWZ ran into. Some people have their Twitter feeds show
up on their blogs (with a ton of obnoxious and slow redirects and
way, the proper internet way, is to invert that model. Host your
messages in a decentralized way on your own indexable, static web
site, and if you feel like dealing with Twitter (or even Facebook),
send it to them as an afterthought. If they die like ping.fm, oh well.
If you visit my main web site today, http://xed.ch, you’ll see that
I’ve just added a new section of recent status messages. I also did
some other changes like finally moving my whole blog system from "dev"
to "live" while leaving a dev version to play with. Hopefully it all
I haven’t hooked the status messages up to Twitter yet, but I may if
the API key circus isn’t too onerous. But first, I want it to work
where it really matters, on my web site. I rewrote the entire system;
basically I edited my Makefile and my main page template, and created
a one line sed program. It’s not really that hard! I’m going to see
how it goes. I’m also going to think about best practices for posting
links now that URL shorteners are clearly evil (including goo.gl and
especially the non-optional t.co and facebook hijackers).
If you’re an actor or interested in writing fiction, it’s a good idea
to read books like
Every Body Is Saying by former FBI agent, Joe Navarro. This book is
about how to read body language. The FBI is interested in this for the
same reason they’re interested in
polygraphs. Like polygraphs,
this stuff doesn’t always work reliably. Nonetheless, I still find
non-verbal communication fascinating. If you’re writing fiction, its
lack of reliability doesn’t matter. If a certain hand gesture
supposedly indicates a certain mood, when you’re trying to convey that
certain mood you can throw in the plausible gesture as a bonus. It
doesn’t matter if it’s really accurate; what matters is if it rings
true to you and, if so, maybe your readers will feel the same. Anyway.
Enough of my fiction writing tips.
Palm In, Serious
While reading this book I came across this passage which I took
special note of.
In many countries throughout the world, finger pointing is viewed as
one of the most offensive gestures a person can display. Studies show
that people don’t like it when someone points a finger at them. In
schools as well as prison yards, finger pointing is often the
precursor to many fights. When talking with their children, parents
should be careful to avoid pointing at them while saying things like
"I know you did it." The finger pointing is so distasteful that it may
actually divert the child’s attention from what is being said as they
process the hostile message of the gesture.
Lumping schools in with prisons always makes me smile, but there was
something deeper about this. After giving it some thought I wondered
what situation I had been in where someone pointed at me. It is pretty
rare. But then it came to me — Trump.
Palm Down, Serious
I had this image in my head and wondered if it were valid. So I did a
very rough experiment. I did a Google Image search involving a person
and the word "pointing" (
like this) and for
each of the first N images in order, I classified the types of pointing the
person was doing. The quantity N varied based on the quality of the results. I
kept recording until I had a decent sample or until the images stopped being
sensible (a lot of pointless "other" images).
Palm In, Smiling
For some of my controls like "fred rogers", "beyonce", and "bon jovi", etc. it
was clear that there really wasn’t much pointing to be seen at all. But when I
turned to politicians it became quite interesting. I’m again using
my recent enthusiasm for pie charts and a bit
turn that on). You can click the names and
change the chart.
"dalai lama pointing"
"steve jobs pointing"
Click search term to select.
Trump’s narrative never seems to require
facts. This is unnerving for some of us, but that leaves the
conclusion that there is more going on than standard models of
reality. It would seem for this reason that Trump is an especially
interesting subject for analysis of his non-verbal communication. My
conjecture is that being the scion of a wealthy family, he has
naturally internalized subconscious mannerisms of a high status
individual. He exudes the idea that he is our boss, liege lord, or
mafia don. He’s simply the guy who is more important than us. Actual
facts supporting that claim aren’t terribly important. He can get away
with vulgar boorish behavior because it is expected of people who
are less exposed to consequences. For many, apparently, it serves to
remind them that Trump is their natural superior and leader. From my
perspective, not so much. Now enjoy
this video of upper class
twits kicking beggars that has an unusually happy ending.