Thursday, September 25, 2008

MSO affecting 21CN customers

There is a major service outage affecting our 21CN customers, BT are aware of this and engineers are currently working on the fault.

Update to follow.

8 Follow-up Messages (Posted by AAISP Staff):

AAISP Status said...

This major service outage is affecting ADSL1 and ADSL2+ customer on our new pipe only.

AAISP Status said...

A BT engineer is on site and will be replacing a faulty card, hopefully very soon.

AAISP Status said...

The current update is that there has been a "Serious Incident" at Stepney Green. We are led to believe that it may be several hours before service is restored. Sorry, we do not have more details. The impact is that all of our customers on 21CN are off line as are all other ISPs with 21CN links out of Docklands as we understand it.

AAISP Status said...

BT have just confirmed:-

I am writing to let you know that we have had a break-in at our Stepney Green Exchange and it will have an affect on your WBC/WBMC network.

I do not have too many details at present, other than the police are involved and are currently carrying out their investigations.

Please be assured that we are doing everything we can to replace the missing hardware and bring you back into service.

There will be a briefing published shortly with more information on it.

AAISP Status said...

Finally back on line. Thank you all for your patience.

BT briefing shortly before the restoration of service :-

Preliminary investigation suggests that at, 03.44 GMT on 25th September, Stepney Green Exchange in London experienced a suspected breach of security, involving physical break in. This resulted in the theft of 9 network cards and caused service failures on 21C WBC Broadband services for a number of BT customers.

Police, and BT investigation teams, are on site at present.

Restoration of service is dependant on the conclusion of current police investigation on site, the arrival of replacement cards, and the assessment of any further technical damage to the equipment and connections.

BT very much regret and sincerely apologise for the impact of this event and we wish to assure you that we are doing all we possibly can to implement appropriate remedies to ensure both full restoration of service and protect resilience for our customers and services.

AAISP Status said...

To our total disbelief the new pipe is down yet again. We are investigating.

AAISP Status said...

Well, it is back.
Awaiting explanation.

AAISP Status said...

Apparently this second outage later in the day we BT putting in new cards to replace the ones they borrowed earlier in the day that replaced the stolen cards. We are trying to find out why we had no warning of this.