CiaN:MinDMaP

Yes!  We are prepared.

That’s what we are now able to say when people ask us.  Basically we are processing DNS logs, scraping for any of the 90,000 Conficker A+B domain names.  As of tomorrow that won’t necessarily work, as it will be generating 50,000 new domains every day.

What we are doing is comparing all the DNS queries that come in with an in memory database of the conficker domains (very fast), then cross-indexing those IP’s / timedate stamps with user records.  Once done, this information is injected into our existing anti-abuse system.  While I can’t say what happens then, let’s just say that our existing process (people, machines, etc) deal well with viruses, and worms.

My current (un-verified) estimate is that there are anywhere from 0.001 – 0.002% of our customers infected.  This is lower than expected, as I think Canada makes up for about 5.8% of the entire 3 Million hosts, which would work out to a national average of 0.005 – 0.006 %.  I entirely attribute this to some of the super secret “we’ll take care of you” anti-abuse systems we’ve put in place for consumers.

I never realized how hard it is to talk about stuff like this, and remain confidential

Tags: botnet, computers, dns, systems, work
Posted in Personal | | No Comments »

So the internet as we know it is extremely broken.  And for some reason know one seems to know about it.  Except us g33ks who run it.  Seriously it’s so broken, that it should be headlining on CNN.  But nothing.

A few months back we got notification from the US Govt that Dan Kaminsky had identified a major hole in the protocol spec for DNS.  It had to do with DNS only usuing a very limited amount of source ports for sending out it’s requests.  Along with this limited number of source ports it uses a 16 bit transaction ID on each of the packets.

Kaminsky had identified that you could force a remote DNS server to do a query for 1.cian.ca, then 2.cian.ca then 3.cian.ca, etc.  Then, while it’s waiting for the real cian.ca to respond, you flood it with a whole bunch of packets to that limited number of ports, each packet guessing the 16 bit transaction ID.  Given the limitations DNS had before, you could have a 60% chance of getting it right considering how many packets you were sending.

The real trick was to respond with the information telling it what 3.cian.ca was, but then adding (as in the DNS spec) “Additional Information” which had authoritative records for the NAMESERVERS for cian.ca.  So even if your target had cached the nameservers for cian.ca already, you can re-point that domain to any IP you wish, for whatever target nameserver you wish.

So think about targeting AOL customers.  Then think about overwriting microsoft.com, or cnn.com, or how about royalbank.com.  Now it’s getting scary.  And up until we all patched, that really wouldn’t be very hard to do.

So everyone patches.  It’s the biggest co-ordinated upgrade in history.  And we were all pretty quiet about it.

Now what is happening is the source port is being randomized for queries.  So now you have about 64 thousand ports to guess in addition to the 16 bit number.  This is about 2 to the 27 ish possibilities.

Dan Kaminsky has now also sucessfully demonstrated that with a Gig connection, and two attacking hosts, he can redirect nameservers within about 10 hours (instead of seconds) now.  That’s only using 2 hosts.

What would happen if you say…had a botnet of a couple million?  Divide them up, assign a couple per port, and have them co-ordinate an attack on a nameserver?  You could literally redirect at will.

And I’m sure it’s being done right now.  Think it will be a while before I log into paypay, ebay, gmail, banking…..oh shit….how the hell can I do that?!?!?

Now…CNN, CBC, all you guys, WHERE THE FUCK IS THE NEWS ABOUT THIS?!?!?!?!?!?!?!

Tags: cnn, dns, hack, kaminsky
Posted in Business, Personal, Sysadmin | | No Comments »

So here I am working at 1:30 am. Man this job is fun. Today I was slicing a disk on a brand spanking new Sun box that I’m building from scratch. And to boot, no one to date seems to have been able to get Solaris Liveupdate to work on their machines. Why? I’m guessing that it’s probably because some of the disk slices were setup incorrectly. I looked at some machines today and they were attempting to use, or not using at all the reserved backup slice 2.

After I setup all the slices properly, I created a mirror accross the two disks. Sounds geeky but man it’s fun.

Just tonight I put a few DNS servers that I’ve been building for the last two weeks live. They are chugging away nicely. I’m very happy actually with how smoothly it all went – I had to port a web application that is used to make changes to an Oracle database, which holds all of our 15000 DNS zones. Then I had to make sure that this script which runs on each nameserver that generates all the zones from the db worked.

After that, I had to get routing setup for the two virtual OSPF IP’s that I was pulling from the current live servers in the other data center. It was really neat to just be able to turn up my interface, and pull traffic from two live servers on the other side of the city.

After coming from a world where you have either windows, or linux, and seeing a lot of frustrations with some of the ‘covenants’ that Solaris observes, it’s nice to see that it has a nice, solid interface to manage disks, their geometry and setup, and a real way to see what’s going on. As long as you have a calculator handy anyways :-p

Alright, just got my call, did some final tests and everything is a-ok! Bed time!

Tags: disk, dns, liveupdate, mirror, oracle, ospf, slice, solaris, work
Posted in Personal | | No Comments »