Thursday, June 19, 2008
Routing issue affecting broadband customers
We are seeing an issue with routing in the data centre in London that is affecting broadband customers and are looking in to it now. It appears one of our routers is playing up.
Subscribe to:
Post Comments (Atom)
Posts
3 Follow-up Messages (Posted by AAISP Staff):
This appears to be affecting some customers but not all. We are still investigating.
It would appear taht a problem with one of our BGP routers caused some traffic to be routed to a black hole for some customers. This was resolved just before 9am. We are trying to get to the root cause.
Looks like the issue was a backup router getting in to a partly working state. As is often the case with redundant systems - a part of a system breaking totally is fine but when something is partly working it can cause issues. We think we know why the problem happened, being an unexpected side effect of an interface change on a backup router when adding links for our new pipe. These type of changes are normally harmless and always tested on the backup system first anyway, but we may have to rethink the way that is done in future.
As side note - we are well on the way to having completely custom routers in use. We'll be announcing planned work over the next few months where we gradually introduce completely new core routers throughout our network.
Post a Comment