sniffer config's

[rn4it]

I need to come up with a solution that is inexpensive, but will be able to monitor and set off alarms in a multi-switched environment. I need to be able to capture suspicious traffic and alert Network Mgmt team. there are 20+ switches, I would like to be able to monitor as much traffic as possible.

I have seen solutions which involve taps on the core switch, going to a monitor. However, this doesn't allow us to monitor traffic that stays on the same switch.

any ideas?
thanks
J

 

[johnny99]

Finding a inexpensive solution to that will be more then difficult.

ISS have solutions for this and the products they have are great but $$$$

If you had a single core switch where all data had to pass you could mirror all trafic to a port and scan it all at that mirror port.

But the problem already starts here. A 6808 can handle 128Gbps and 96Mpps (just multiply by 2 if you have a 6816) so if that switch is loaded with just 1% you would not even be able to mirror all the trafic to a single Gbit port. Having 20+ of them "My God".

You would need some kind of agent connected to each switch and then get all the interesting results to a single console for monitoring.

Do you realy need this kind of monitoring?

I can only thing of a few installations that realy have this kind of needs and they don't ask what it cost (eg. NSA and Swift)

 

[rn4it]

Yep, I agree 1 solution is to throw $$$$ and come up with a solution. However, I have been instructed to come up with a solution that will allow me and my team to be able to answer any questions re: the Network. If anything weird happens to be able to prove that it was or not network traffic or access via the network that caused the disruption. At the same time, I have been asked how do companies that don't have the $$$ handle such a task? Do they rent the equipment and expertise in a reactive capacity, have a small amount of eqipment that gets rotated throughout the network, nothing at all,etc or what?

I guess what I'm looking for is answers from Net admins in midsized companies, that haven't thrown $$$ into a solution or consults that have worked in such an environment.

Thanks for your input.
J

 

[johnny99]

"to be able answer any questions re. the network" especialy in the past will be a problem.
From a technical point it can be done (as it could be done to send people to the moon and get them back alive, but it will cost a lot.

In our installation where we have a high security level we have a BlackDiamond 6808 as the core switch and some 24+ Summit24 at the different floors.
4 times a year we have a external company that hook up some kind of LAN monitor for 30 min and give us a report on what they have seen.
We have a ISS system from RealSecure that we use for intrution detection and penetration testing.
We have opted not to use the gigabit port option for ISS. If I remember right that came at a price around USD 40.000 or more (just to upgrade it to connect to a gigabit port and not a 100Mbit/s port.

Basicly what you need to be able to answer any kind of questions is a system that logs all your network traffic and stores it in a database and then have a tool to analyze the data.
Since you don't know what they could ask you tomorrow about what happend 3 months ago you need to store all information.
Even if you get the software as freeware you would need a bunch of StorageTek PowderHorns with 9940B drives just to store the data. Even in our small shop logging all network traffic for 12 months would run in Peta bytes.

Your way forward (as all of us have to)is to get management to narrow down what kind of answers you need to be able to answer for how long time back.

What they want you to do, is stuff that is done at the NSA but only giving you USD 20.000 to do it.

Management is asking you something that can't be done given the frame given to you.

 

[rn4it]

I agree that what I'm being asked is to attempt to implement a 20K+ solution for a $1, but I do have to do my do diligence in specking out a solution. There is alot of freeware apps that are relialbe(ie snort) for network reporting I just came across cricket, which I will be testing next week and if it works then great it will be give me reports that I can prove that network utilization is not the problem with such and such an application, without spending $$$. That is all I'm asking, and I'm not saying log every packet, just suspicous or corrupt traffic. Even if all we do is keep weekly reports that would be acceptable. I'm well aware of costs and the size of log files could get, but disk space is cheap and backups are wonderful.

 

[johnny99]

What made me "jump" was "to be able to answer any questions re: the Network" because I have had managers who basicly wanted things like that.
A director gets some kind of complaint and he comes to me and ask if user xyz using application 123 at 11:30 to 11:35 2 or 3 weeks ago had some kind of problem.
When you try to tell them that your best answer is that we didn't have any network problems at that time most of them want what they call a "better answer".
As I read what your management want is basicly that: Give me answers to anything I find I out I want answers to even if it happened 6 months ago.

Having stats on switch performance is a lot more simple.

Try to look at MRTG it has support for BlackDiamond switches (if that is what you use)
You can find MRTG at

http://www.mrtg.org

 

[rn4it]

I'm installing cricket right now, it's similiar to MRTG, but our FW vendor will support it to a degree. I still would like to get sniffer traps setup to capture suspicious traffic. Typically, if a Mgr or higher wants to know what happed on the network, it's usually within the week. I myself don't like unknown answers, so I would like to be able to capture suspicious traffic. That way if a device stops talking on the network, I can rule certain things or say what I have seen in the traffic. example is there have been times that an application stops working, everyone spins and points at the network. I look on the switch, and no ports flapping, no changes, no errors, no high utilization. I would still like to have more proof that it was the application or user, since, many people believe those error messages from their application that there are network problems.

 

[johnny99]

I would advice you not to expect too much.
I don't know what switches you have but on our 6808 we can't monitor all the traffic since the fastest port we have is 1 Gbit/s and we have 40 of them and 48 10/100Mbit/s ports. The mirror function only mirrors all traffic to one 1Gbit/s port but it starts dropping frames if the load on the switch is above 1% (or even less)
We can't mirror 5 Gbit/s traffic to a 1 Gbit/s port :-(

Another great too to snif around is Sniffer from Nai (it was Network General some years ago) but it's also a $$$ product.

 

[Comstocb]

Johnny99 pretty much nailed it. If you want granularity and especially if you want to archive to go back, cubic dollars will be spent. I run a finisar THG and then tap all major links and a few interesting ones but that will only allow me to capture traffic if I am aware of a problem. I have zero archive ability and the cost just makes it out of the question. Trending with MTRG or Cricket is good because you can at least identify if a major traffic spike or something similar occured at the time in question though you will have nothing to analyze why. When the network is questioned, ask if the problem is repeatable or if its constantly occurring. Then you can set up to capture and analyze otherwise you'll be wasting your time 99.9% of the time. Tools are great but they can never replace organization (I almost micro-manage everything about the LAN down to even how the cables are run and secured. It pays off big when problems occur because you know EVERYTHING about the environment)and a true understanding of your environment and how it works, read protocols and their behaviors. Let me know if this helps

Brian

 

[rn4it]

I know what you guys are saying, but here is the catch. $$'s are tight, management wants reports, fine, we can do that via mtrg or cricket. They also want us to be able to tell them what is going on the a perticular application, from a Network persceptive. There have been a number of times where we either need to use a sniffer or would have been nice to have captured it on a sniffer. Money is tight, and I have to know what does a company do with no budget or can't afford a finisar, sniffer pro etc solution. What do they use? What do they do? This question still hasn't been answered, so please don't say throw $$$ at it. Say, they rent equpment, contract a person out, use X tool, etc. I don't want to sound unappreciative, but I think I'm asking a fairly straight forward question. If I had it my way I would of had a sniffer tap solution in place by now, and have it paging me with alerts, for a fairly inexpensive price(approx$20K). I have talked with our Finisar rep, but I can't get a sign off, until I hear from that Netadmin in some 3rd world country stating they use finisar.

 

[Comstocb]

I guess we are trying to say management isn't going to get something for nothing. In network design, I always include the cost of network troubleshooting equipment when specing out a new network infrastructure. That way I don't get the accounting roadblock when it becomes critical....

That said, A laptop and a free software sniffer (ethereal) plus mrtg/cricket are the tools you need. Management may not like what they get but you can't fix a new Corvette without expensive diagnostic tools, just like networks. SNMP is only going to give you general information and trending data. You want to be able to tell them whats really going on, hardware sniffers and taps, end of discussion. Nothing else even comes close in granularity. If your environment is small though, I would tackle it with the free stuff mentioned above. The real problem with software sniffers is that they don't capture things like CRCs and will start to drop packets with on 6-8% utilization on a 100 full link. Try and capture on a gig link......you see my point. I hope this helps. See if the free stuff meets your needs, if not, your next job is selling management on the tools so THEY get what they want.

Good Luck

Brian

 

[johnny99]

The short answer to your request is that it can't be done for USD 20.000

As I wrote before: What they want you to do, is stuff that is done at NSA but thwy will only giving you USD 20.000 to do it.

Management is asking you something that can't be done given the frame given to you.

If you find a solution to your task costing USD 20.000 a lot of us would love to hear about it, but even USD 200.000 wouldn't get you what you tell us your managers want from you.

Thank God that my CIO isn't that way, and I also think my people think the same way about me.

 

[StevenWilliams]

Hi guys,

All I can add is what I do already.

* use MRTG / RRDTool to monitor links of interest and measure bandwidth utilisation.
* use Epicentre to monitor / manage switch events and thresholds and use the real time monitoring to regularly sweep across switches and look for errors
* use Whatsup Gold for any other SNMP or ICMP monitoring
* use Kiwi Syslog Daemon to dump all logs to syslog
* use Snort on our DMZ via vlan mirrors, and run snort on a laptop via port mirrors to randomly select links and analyse for a 24 hours
* use Sniffer Pro LAN for any analysis

Unfortunately the Netflow implementation is really a false sales pitch as we found out. If it was working properly, we would also use Crannog Netflow Accounting to pull netflow packets as well.

Cheers

Steve

 

[Comstocb]

Hello Steve
I was looking at the Netflow information out of curiousity and noticed in their docs that it is meant to work with Cisco equipment by polling the Cisco Netflow information. Why would you use this with Extreme equipment and if it was possible, what was false about the sales pitch?? Just trying to un-ignorify myself. Thanks in advance for your time.

Brian