BananaGlee. I just love saying that word ;)

So, was reading up on the NSA backdoors for Cisco and other OSes,, and got to thinking about how the NSA might exfiltrate their data or run updates…It’s gotta be pretty stealthy, and I’m sure they have means of reflecting data to/from their Remote Operations Center (ROC) in such a way that you can’t merely look at odd destination IPs from your network.

This got me thinking about how I would find such data on a network. First off, obviously, I’d have to tap the firewall between firewall and edge router. I’d also want to tap the firewall for all internal connections. Each of these taps would be duplicated to a separate network card on a passive device.

1) eliminate all traffic that originated from one interface and went out another interface. This has to be an exact match. I would think any changes outside of TTL would be something that would have to be looked at.

2) what is left after (1) would have to be traffic originating from the firewall (although not necessarily using the firewalls IP or MAC). That’s gotta be a much smaller set of data.

3) With the data set from (2), you’ve gotta just start tracing through each one.

This would, no doubt, be tons of fun. I don’t know how often the device phones home to the ROC, what protocol they might use , etc…

If anyone has any ideas, I’d love to hear them. I find this extremely fascinating.


The Internet Almost Crashed!

Yeah, it is true. I guess some programming errors are more serious than others, so lets give these guys a break: I also suppose the dark clouds gathered for all the recent DDoS characters, too.


Cisco: We know IOS rootkits can be made – harden your system

cisco has released an updated version of its cisco security response: rootkits on cisco ios devices document after the eusecwest presentation of mr. sebastian muniz (core security).

hardening, best practices etc, it appears.

thanks Sunshine. for pointing this on mailing lists.


Why coding after a long drinking night is not a good idea

I’d love to hear the background story behind this one:

[CiscoWorks IPM] version 2.6 for Solaris and Windows contains a process that causes a command shell to automatically be bound to a randomly selected TCP port.

Why on earth? And why a random port?

And if you’re still wondering, yes – it’s a remote root shell with no authentication

Remote, unauthenticated users are able to connect to the open port and execute arbitrary commands with casuser privileges on Solaris systems and with SYSTEM privileges on Windows systems.

Cisco is being cruel and only disclosing the technical info. Common Cisco, share the juicy parts! We want Full Disclosure!


eWeek: Estonian Cyber-War Highlights Civilian Vulnerabilities

i posted a column on eweek on what critical infrastructure means, looking back at the estonia incident.

they edited out some of what i had to say on home computers and their impact as a critical infrasrtcuture, but hey, word limitations.,1895,2166125,00.asp

Gadi Evron,


Google, Service Providers and the Future of P2P

in a non-operational nanog discussion about google bandwidth uses, several statements were made. it all started from the following post by mark boolootian:

> cringley has a theory and it involves google, video, and oversubscribed backbones:

in the discussion, the following statement was made by rodrick brown:

> the following comment has to be one of the most important comments in
> the entire article and its a bit disturbing.
> “right now somewhat more than half of all internet bandwidth is being
> used for bittorrent traffic, which is mainly video. yet if you
> surveyed your neighbors you’d find that few of them are bittorrent
> users. less than 5 percent of all internet users are presently
> consuming more than 50 percent of all bandwidth.”

from there it went down-hill with discussion of the future, with the venice project (streaming p2p for tv), etc. being mentioned. some points were raised about how isps currently fight p2p technologies and may fight these new worlds of functionality, denying what the users want rather than work with them, citing as we have seen above that today, a very small percentage of internet users account for about 50% of all internet traffic. that of course, will increase dramatically in the future — it is where the users want to go.

the isps inhibit this progress, just like in my opinion a bad security “guy” or “gal” would try to prevent functionality from their users as part of their security strategy, rather than work with their users and enable functionality first.

in this discussion, randy bush (who i have had my share of strong disagreements with in the past) said the following, which is admirable:

> the heavy hitters are long known. get over it.
> i won’t bother to cite cho et al. and similar actual measurement
> studies, as doing so seems not to cause people to read them, only to say
> they already did or say how unlike japan north america is. the
> phenomonon is part protocol and part social.
> the question to me is whether isps and end user borders (universities,
> large enterprises, …) will learn to embrace this as opposed to
> fighting it; i.e. find a business model that embraces delivering what
> the customer wants as opposed to winging and warring against it.
> if we do, then the authors of the 2p2 protocols will feel safe in
> improving their customers’ experience by taking advantage of
> localization and proximity, as opposed to focusing on subverting
> perceived fierce opposition by isps and end user border fascists. and
> then, guess what; the traffic will distribute more reasonably and not
> all sum up on the longer glass.

it has been a long time since i bowed before mr. bush’s wisdom, but indeed, i bow now in a very humble fashion.

thing is though, it is equivalent to one or all of the following:
-. eff-like thinking (sticking to the moral high-ground or (at times!) impractical concepts. stuff to live by.
-. (very) forward thinking (not yet possible for people to get behind – by people i mean those who do this daily), likely to encounter much resistence until it becomes mainstream a few years down the road.
-. not connected with what can currently happen to affect change, but rather how things really are which people can not yet accept.

as randy is obviously not much affected when people disagree with him (much the same as me), nor should he be, i am sure he will preach this until it becomes real. with that in mind, if many of us believe this is a philosophical as well as a technological truth — what can be done today to affect this change?

the service providers are not evil — they do this out of operational necessity and business needs. how can this change or shown to be wrong?

some examples may be:
-. working with network gear vendors to create better equipment built to handle this and lighten the load.
-. working on establishing new standards and topologies to enable both vendors and providers to adopt them.
-. presenting case studies after putting our money where our mouth is, and showing how we made it work in a live network.

staying in the philosophical realm is more than respectable, but waiting for fussp-like wide-adoption or for sheep to fly is not going to change the world, much.

for now, the p2p folks who in most cases are not eveel “internet pirates”, are mostly allied whether in name or in practice with illegal activities. the technology isn’t illegal and can be quite good for all of us to save quite a bit of bandwidth rather than waste it (quite a bit of redundancy there!).

so, instead of fighting progress and seeing it [p2p technology] left in the hands of the “pirates” and the privacy folks trying to bypass the firewall of [insert evil regime here], why not utilize it?

how can service providers make use of all this redundancy among their top talkers and remove the privacy advocates and warez freaks from the picture, leaving that front with less technology and legitimacy while helping themselves?

this is a pure example of a problem from the operational front [realm] which can be floated to research and the industry, with smarter solutions than port blocking and qos.

it’s about progress and how change is affected and feared, not about who is evil. it is about who will step up and make a difference, and whether business today is smart enough to lead the road rather than adapt after the avalanche has already fallen.

gadi evron,


CCC: Router and Infrastructure Hacking

1. at ccc last week raven alder gave a talk on the subject (router and infrastructure hacking), which was pretty neat!

i figure some of you may enjoy this. i hope the video for her talk becomes available soon.

2. there was also a lecture on sflow, by elisa jasinska:
presentation and paper:

3. i do wish the talk on how ccc set up their multiple-uplink gige network for the conference was filmed, i call this type of “create an isp in 24 hours”, in a very very hostile and busy environment such as at defcon or ccc “extreme networking”.

they got their own asn for 4 days. set up a hosting farm, surfing, mass wireless, etc. for users, and what-not. discovered a wireless network vulnerability, a router dos with nexthop memory issues, etc.
not to mention having to fight off ddoss non stop, fake aps, thousands of active and abusive users and bgp (i really liked their presentation on ripe’s bgplay – very cool stuff - ).

3000 end points. 1.6 gigs up, 1.0 gigs down.

their slides are up at:

as mentioned before, ccc itself was very good and a lot of fun, there are many other presentations and videos available for download:

gadi evron,


Drop zones and an intelligence war

in this post ( ), fx describes a drop zone for a phishing/banking trojan horse, and how he got to it.

go fx. i will refrain from commenting on the report he describes from secure science, which i guess is a comment on its own.

we had the same thing happen twice before in 2006 (that is worth mentioning or can be, in public).

once with a very large “security intelligence” company giving drop zone data in a marketing attempt to get more bank clients (“hey buddy, why are 400 banks surfing to our drop zone?!?!)

twice with a guy at defcon showing a live drop zone, and the data analysis for it, asking for it to be taken down (it wasn’t until a week later during the same lecture at the first isoi workshop hosted by cisco). for this guy’s defense though, he was sharing information. in a time where nearly no one was aware of drop zones even though they have been happening for years, he shared data which was valuable commercially, openly, and allowed others to clue up on the threats.

did anyone ever consider this is an intelligence source, and take down not being exactly the smartest move?

it’s enough that the good guys all fight over the same information, and even the most experienced security professionals make mistakes that cost in millions of usd daily, but publishing drop zone ips publicly? that can only result in a lost intelligence source and the next one being, say, not so available.

i believe in public information and the harm of over-secrecy, i am however a very strong believer that some things are secrets for a reason. what can we expect though, when the security industry is 3 years behind and we in the industry are all a bunch of self-taught amateurs having fun with our latest discoveries.

at least we have responsible folks like fx around to take care of things when others screw up.

i got tired of being the bad guy calling “the king is naked”, at least in this case we can blame fx. :)

it’s an intelligence war people, and it is high time we got our act together.

i will raise this subject at the next isoi workshop hosted by microsoft
( ) and see what bright ideas we come up with.

gadi evron,


Internet Security Operations and Intelligence II

isoi 2 is finalized. the schedule and agenda can be found here:

i am going to do my best to release some of these presentation publically after the event (if the authors agree), but it is not likely.

some public feedback will be relayed from the workshop.

gadi evron,


QoS and bot traffic

i am starting a discussion in the relevant groups on this subject, to try and come up with some suggestions and to-do items we can follow up on, or maybe even better – find another solution.

networks require a means by which they can control their botnet population. yes, “curing” the problem is great, but it won’t happen in the near future.

obviously, having isp’s call even one customer to remove infections doesn’t work (costs significantly more than the subscription fee per attempt) and people just get re-infected.

i am looking to utilize proven technology to be able to reduce the cost of what a botnet can do.

if botnet traffic is detected, even by not very sophisticated technologies such as simply checking for email sent from dynamic ranges or netflow data, it should be possible to use routing technology to “mitigate”.

qos can limit the traffic these bots can utilize much like it would p2p users in most isp’s today. these users are already of limited traffic due to the effects of the bot.

how can this be done using today’s technology? does it require re-design of hardware or new systems to be designed? i hope to find out and get a proposal ready,

gadi evron,


ISOI II – a DA Workshop (announcement and CFP)

the second internet security operations and intelligence (isoi) da workshop will take place on the 25th and 26th of january, 2007. it will be hosted by the microsoft corporation, in redmond wa. an after-party dinner will be hosted by trendmicro.

this workshop’s main topic is botmaster operational tactics – the use of vulnerabilities and 0day exploits in the wild. (by spyware, phishing and botnets for their businesses).
secondary subjects include ddos, phishing and general botnet subjects.


Internet Worms and BGP Storms

i surfed the web today, and reached jose’s blog. he covered a paper called:
“is bgp update storm a sign of trouble: observing the internet control and data planes during internetworms”
matthew roughan, jun li, randy bush, zhuoqing mao and timothy griffin.

you can find it here:

the paper’s abstract:

there are considerable reasons to wish to understand the relationship between the internet’s control and data planes in times for stress. for example, the much publicized internet worms—code red, nimda and sql slammer—caused bgp storms, but there has been comparatively little study of whether the storms impacted network performance. in this paper, we study these worm events and see whether the bgp storms observed during the worms actually corresponded to problems in the internet’s data plane. by processing and analyzing
two datasets from ripe, we have found that while bgp update storms occurred in all three worms, the performance of the data plane degraded during the slammer worm but did not during the code red and the nimda. no direct correlation should be drawn between the degradation of the internet data plane and the occurrence of a bgp update storm—it may not be a sign of trouble but a sign of the internet control plane doing its job. (more…)


Cisco Systems IOS GRE Decapsulation Fault

i would like to draw your urgent attention to a couple of securiteam articles:
the advisory, released by fx:

“cisco systems ios contains a bug when parsing gre packets with gre source routing information. a specially crafter gre packet can cause the router to reuse packet packet data from unrelated ring buffer memory. the resulting packet is reinjected in the routing queues”.

this is the cisco response:
original url:

if you are not into routing, this is what gre is:

gadi evron,


Courtney Love explains BGP

it is not often we get to have some fun while dealing with the realm of bgp. that said, you can get a good rotfl and learn from this surprisingly informative post:

if you like, look for other posts there, such as “don king on ip access lists” or “gary coleman on priority queuing”. whatever you do, read this. :)

have fun. :)

thank to twi for this link.

gadi evron,


BlackHat USA 2006 Scandal with Michael Lynn? Not Quite.

yesterday (now two days ago), fx, mumpi (see, i remember!), nicole, dan kaminsky and myself went to the cisco black hat party. with us was the ciscogate renowned michael lynn.
we were fx’s guests, as he kindly invited us (great guy who knows how to have fun, and unlike most people, was as honest and blunt as i usually am when we talked, gotta respect that).

we went to the party, registered, said hello to a couple of cisco employees who knew who each and every one of us was (bouncers), a club bouncer, and entered the party. one of many community fun after-parties that come with these conferences.

so far so good. cisco was fun and the party was great. mike spoke with many cisco guys (no hard feelings on either side, it seems, we’re all in the same industry) and we even got our pictures taken together.

a nice older lady, who kept smiling, stood with us in line as we got into the party, etc. was known to us as a reporter. i was next to mike when he said “hello, how are you?” and moved on, not looking to talk with the press yet still being polite.

she watched as mike signed his name.

then, she released this tabloid-like article on, and i quote:
“juniper researcher michael lynn crashes cisco party at black hat”

“the invite-only party last night that cisco held at a nightclub for black hat conference attendees was crashed by security researcher michael lynn, who last year was sued by cisco for revealing a serious flaw in cisco routers.”

and before i get too annoyed, last quote:
“along with some friends, michael lynn, who now works for cisco rival juniper networks, evaded the security checks cisco had put in place for the party, which included a name check and legal identification. lynn and his friends, declaring “cisco owes us a drink,” gleefully posed in front of a cisco sign inside the pure nightclub. once aware the lynn entourage had crashed the party, cisco employees took it in stride.”

not any usual tabloid though, no sir. this was at network world. you can read this article, if you can call it that, here:

inventing a story for fun? trying to get mike into trouble? whatever her game is, this was just low.

there were at least six other reporters there, non of them did this. she did.
ellen messmer, thank you.

as a side-note, that same evening we went in a limo to the zdi party, thumbs-up to tipping point (“a division of 3com”) for the great party! and for fx for knowing how to have fun!

gadi evron,


Internet Security Operations and Intelligence – a DA Workshop

the da workshop will be mostly on the subject of botnets, while touching phishing and ddos.

it will take place on august 10th, hosted by cisco in san jose with a dinner, sponsored by the isc.
participation is open only to members of closed and vetted mitigation and security operations groups.

main lineup:

“bot, botnets, sandbox, impact”
righard j. zwienenberg (norman)

“msrc malware/exploit zero day response – case studies”
greg galford (microsoft)

“the rough road around us in botnet tracking”
jose nazarijo (arbor)

“malcode toolkit profiteering:feeding the trend in m.o. from fame to fortune”
hubbard dan (websense)

case study: ***
levi gundert (us secret service)

“recent bots detection information from microsoft security products”
ziv mador (microsoft)

“security inside the router:how network gear handles ddos attacks”
barry raveendran greene (cisco)

“what keeps us up at night:
new & advanced difficult to mitigate ddos attacks”
darrel lewis (cisco)

“the global infection rate”
rick wesson (alice’s registry)

“phishing and botnets organized crime:
globalization and tehnology intelligence update”
gadi evron (beyond security)

“fast-flux botnet c&c servers – detection & mitigation”
randy vaughn (baylor)

david ulevitch (everydns / opendns)

jerry dixon (dhs – us-cert)

paul vixie (isc)

the web site for the workshop is:

gadi evron,