DMIT noticed the speed issue to China Unicom Guangzhou from LAX Pro CN2 GIA since week ago.

We’ve working on it; it takes time to work with ISP NOC.

The test can only be done during the peak hour.
With the time difference of all parties. The estimate repairs time will be in this week.

Please have a test and leave us a ticket if there is any issue with this case.

https://www.dmit.io/pages/lax-eyeball

CT > DMIT: - AS4809
CU > DMIT: - AS4134 > AS4809
CM > DMIT: - AS58807

DMIT > CT CU CM: AS58807

OpenBeta discount code:

LAX-EB-LAUNCH-NON-MONTHLY-RECURRING-20OFF

LAX.EB:

- Enabled DDoS Always-On for all IPs.
- Traffic counter has been reset.
- Strict rule applied. Inbound network performance might experience impact.
- Local Latency slightly increased.

Attack Model:
A) UDP
Sending 3~10Mbps DNS to each IPv4 in the current /22.
B) TCP
Tiny TCP ACK from actual running Internet server for 1Mbps per IPv4.

Local latency increased because the Internet inbound routing has been redirected to San Jose. There are about 6ms between them.

This is a temporary solution. DMIT has already purchase hardware to upgrade Los Angeles as well.

SJC Maintenance

Start time: April 13, 14:30 EST
Estimated completion time: 30 minutes
Purpose: Switch from KRBD to LibRBD.

If the VM does not come online automatically after completion, please wait patiently or start it by yourself in the control panel. If the current VM is read-only, please reboot it in the panel to recover it.

The inbound from China to DMIT is remained the same.

The hardware and circuit scale up will be ready between late April ~ May.

Observed IO error from Ceph in Hong Kong location.
It has been resolved.

HKG Maintenance

Start time: UTC+8 2024/04/19 02:00
Scope of Impact: All HKG hosts
Purpose: Upgrade NICs to CX5 Dual Port 100G, resolve high load issues on specific nodes, and upgrade server firmware.
Estimated Completion Time: Approx. 30 minutes per host

All completed. The VMs are booting in sequence.

DIRECT MESSAGE FROM RETN.NET [HKG]

Failure Notification

Description:
Our regional capacity is significantly reduced due to submarine fiber cuts in the Asia region.
During peak hours, multiple destinations may experience overutilization, resulting in packet loss and increased RTT.

Currently, we have registered the following emergency cases:
- Subsea fiber cut on Hong Kong - Taipei section. Started: 15.02 | ETR: 30.04
- Subsea fiber cut on Milan - Singapore section. Started: 24.02 | ETR: 19.04
- Subsea fiber cut on Hong Kong - Singapore section. Started: 10.03 | ETR: 13.05.
- Subsea fiber cut on Hong Kong - Taipei section. Started: 13.03 | approximate ETR: April END.
– Subsea fiber cut on Hong Kong - Singapore section. Started: 08.03 | ETR: 18.05.
- Subsea fiber cut on Hong Kong - Seoul section. Started: 03.04 | ETR: Unknown.

- Emergency situation on the Marseille - Singapore section. Start: 19.04 | Expected recovery: Unknown
- Emergency situation on the Sofia - Singapore section. Start: 19.04 | Expected recovery: Unknown

China Unicom (the Claimant) was unsuccessful in substantiating their claims in the arbitration case against DMIT (the Respondent) for the provision of services AS4837 and AS10099, valued at half a million US Dollars. Ultimately, it was determined that CUA should reimburse DMIT $x2,880.xx(5 digits) USD in the practical terms.

Initially, DMIT had entered into a significant contract with CUA for close to half a million dollars with the aim of joint development and exploring new market opportunities.

However, our plans changed due to unforeseen issues. CUA failed to fulfill the agreed-upon services and unilaterally altered the order, leading to the failed of DMIT's operations in San Jose and degrading our service quality in Tokyo.

CUA denied all liability and sought an arbitration ruling to compel DMIT to reimburse CUA for the full contract value.

As a leading provider of optimized routing solutions from the US to China, DMIT is disheartened by these events and the unreasonable business practices adopted by CUA. We regret to inform that we probably will no longer be working with CUA moving forward.

DMIT wants to share this story to all our partners; stay alert.

DMIT observed CN2 end user routes is now bypassing AS23764 since 3 days ago.

AS4809 is customer of AS1299 in US and EU; but AS23764 is not. This lead to the latency increase recently. DMIT will try our best to fix this ASAP.

Standard.inet.0: 946707 destinations, 2465729 routes (946093 active, 0 holddown, 467099 hidden)
+ = Active Route, - = Last Active, * = Both

58.32.0.0/17 *[BGP/170] 11:39:07, MED 154110, localpref 100
AS path: 174 23764 4809 4812 I, validation-state: unknown
> to _ via _
[BGP/170] 3d 22:49:21, MED 0, localpref 100
AS path: 1299 3356 23764 4809 4812 I, validation-state: unknown
> to _ via _

Temporarily detour CTGnet CN2 in Hong Kong for Premium customer, because of Subnet DDoS attack.

There are 0.5~1Mbps NTP to 4 of our subnets in Hong Kong which is about 700M~1Gbps inbound attack which is almost congested our network and the peer between AS10099 and AS23764.

Already ask CTGnet to check status of their Anti-DDoS system.

Resolved.

Observed HKG EB network issue; working on it. It takes time to work with ISP to fix that out.

DMIT is working with ISP on solving the problem. It might takes a long while.