Most Data centers will have two pipes or more, for redundancy, say if they're in Texas, having one go to Chicago and the other to LA or NY.... So as possible as it is, because a large amount of people still can get to it, I'd still think to look at the ISP end first. Users should contact their ISP only if having problems because if not, they'll show it as being fine. Now the admins should alert the web host as well to a problem, so they can investigate, but something like this unless you're on the inside isn't as easy to track down.

Basing this off the experience I have with working at an ISP previously and with out the networks my company and it's customers use, which are scattered across the US, and several other countries...having some locations store different servers for Apps and such.
[Reply]

You really want to talk to someone who can log on to the router right before the server and do some testing. That seems to be where the problem is at.
[Reply]

My BGP table is bigger than your BGP table
[Reply]

Agreed, more or less. Since the problem is manifesting itself at the 1st upstream router, the problem, statistically, is local to the facility. That is, unless they do physical layer shunting across the US, which is very costly and make little or no sense.

The problem could be a myriad of things from a bad network interface on the server to a HW or SW issue on the upstream device (router or switch) or internal DNS configuration issue or similar.

There is always the possibility that the server is running IP chains or something similar and that the config is screwed up in some way, but that's way far fetched base on what I know.

Like you, I have load of network engineering experience plus telecommunications OS design, work on networking standards committees, and a prior life as a network programmer and college professor. That doesn't mean that either of us knows squat about this problem, however.
[Reply]

This could be the start of a networking banter thread :-)
[Reply]

Duuuuude! :-)
[Reply]

Holy **** you guys lost me at hello :-) LMAO
[Reply]

I would contribute a few packets!
[Reply]

mmblz, hope some of these thoughts end up helping out with the problems that are going on. Keep us updated and we'll keep throwing random crap your way.
[Reply]

Julian and I talked on the phone. We think that the problem is isolated to the configuration to iptables on the server. iptables is a security program (more or less) and it seems to be logging blocked IPs that are corresponding to at least some of those having problems. He's calling the company that owns the server now.
[Reply]

I know this doesn't help with the solution, but it's interesting that both admins that can't log on come from the mid atlantic area and are about two hours apart.
[Reply]

couldn't reach the host yet.
i think the problem might have something to with csf (a program i hadn't heard of before) :-)
[Reply]

Interesting

http://www.configserver.com/cp/csf.html
[Reply]

yeah i found a readme, etc
frank and dave got blocked for typing a bad htpasswd 5 times (lame), and i found the command to whitelist them
now have to figure out why random users would be blocked, or why traceroute would trigger blocking
[Reply]

ssswwwweeeeeeettttttt!

Way to go, Julian.
[Reply]

Must have been a senior moment for them. :-)
[Reply]

:-):-)

What language is this?
[Reply]

It's a language the some crazy people in an asylum made up to pass the time. :-)
[Reply]

see if it works next time you have a chance...
[Reply]

no dice


julian....when the site first went live it kept prompting me for a password. I just hit continue or entered a random password. It always rejected it....could I have the same problem as Frank and Dave?
[Reply]