Don't you just love it....
 

When the crap gets sticky, and your boss is the only person that can advise because you're in unknown terratory, and he buries his head in the sand because he doesn't want his neck chopping off.

Who's got an axe? 'Cos I'm about to start chopping!

(Critical problem across 2 sites, looking like a viral infection causing 2 clients to suffer downtime, means they're loosing money, fast, no-one has encountered the situation before, no-one is willing to help with it either... damn I love my job!)

Sounds like a cue for more "geek talk" :lol:

Take the initiative and sort it yourself :thumbsup:

Advise boss in writing (email) of situation, and explain that further to input from him you will be doing x, y and z where x, y and z are the ways you see to deal with this.

This covers your back

David

Yup, if no-one will help you, just say what your gonna do. They know the problem, they want it solved but refuse to do it themselves or at elast advise/help.

Then go ahead and do it. It it messes up, then its not really your fault, since you received inadequate support.

Do you have any colleuges to talk to about it who might impart their advice?

Matt

Cheers for the words guys, and I did that before posting to the org ;) (including CC'ing the board of directors)

You can tell how much I care about clients loosing money, posting on this forum whilst dealing with the issue :lol:

alpinestarhero, nope, there is NO-ONE in the company that can advise. I'm litteraly in unknown terratory... dealing with a live virus, for which there is no patch, and trying to keep systems running. YAY!

Or at least, I'm treating what I'm seeing as a virus, until I have more information about the things I'm seeing.

Is getting them to pull the plug on what they have and getting "clean" kit to them an option?

Would allow you to ship the infected stuff home and inspect in a clean-room environment

David

Nope, live environment. To use resilience will require a 3 hours downtime in itself, and by now is probably already infected too.

The system requires t'interweb access at all times due to the nature of business. Nice thought though.

I'm dealing with it, slowly & methodically :)

Teching to suck eggs but.....

1) Assuming a firewall between the infected boxes and the outside world, log outgoing stuff

2) Shut down all legitimate services

3) Phone your missus - tell her you will see her at the weekend

4) Look at the logs to see what is trying to connect to what

5) Google for those ports etc

6) Make the rest up as you go along

David

Point 3 is the most important and easily overlooked.

Might I add another point.

7. Ring Dominos or some such other take away establishment and order provisions.

If that was me I'd be pulling the plug live system or not. Sod struggling to keep things up and running, Pull the plug get the pox cleaned up and then off you go again.

I've lost count of the number of times stuff like this happens at work and the managers insist on keeping the system up. When in hindsight it reveals that had we pulled the plug and cleaned the mess up the systems would have been down for less time.