/var/log/DMIT-NOC.log
4.67K subscribers
189 photos
6 files
117 links
Download Telegram
There will be an unscheduled maintenance in the coming hours. 

Work scope: Perform an upgrade on DMIT's Los Angeles core router. 
Impact: 2 hours network downgraded, up to 1 hour network disconnection.
NTT Maintenance.
RETN Maintenance

Planned work on RETN network to upgrade router's software.




New start time: 20.09.2023 20:00 UTC
New end time: 21.09.2023 00:00 UTC
New outage: 4 h

Location: China, Hong Kong
DMIT plan to do following network architecture changes:

New Layer 3 support on hosts, router, access switches with current virtualization hypervisor.
1. Disable ARP and ICMPv6 broadcast in LAN network.
2. Isolated VM domain.
3. Enable IPv6 SLAAC.

Benifit:
1. Reduce broadcast packet amplification in kernel, bridge, and network infrastructure.
2. Prevent broadcast packet uses unnecessary interruption.
3. Improve over all performance. ( Less interruption, less CPU usage in both hypervisor and guest VM. )
4. Offering stateless IPv6, and IPv4 DHCP.
5. Reduce load of router control panel.
6. Reduce ARP traffic consumption on guest VM.

DMIT will apply this based on the following order:
SJC TYO HKG LAX

Expected behavior:
1. Possible disconnection.
2. IPv6 unreachable. ( reboot & cloud-init support requires to automatic update. )
3. IPv4 configuration become /32 (previously /24); gateway is not in the same subet of address.
4. IPv6 configuration become /64 (previously /48); accept_ra = 1 or 2 is required; gateway become link local.
/var/log/DMIT-NOC.log
DMIT plan to do following network architecture changes: New Layer 3 support on hosts, router, access switches with current virtualization hypervisor. 1. Disable ARP and ICMPv6 broadcast in LAN network. 2. Isolated VM domain. 3. Enable IPv6 SLAAC. Benifit:…
For SJC Client ONLY

Due to this change; some guest systems (outdated CentOS 7 etc..) might not install the gateway route.
Please change your system network setting to DHCP to resolve this issue.

Only apply this when your VM cannot get the gateway address for now.

========

This will be resolved by adding user data into Cloud-init; (Within a day.)
It adds the default route with the device route by runcmd.

The DHCP is the solution you currently cannot access the Internet; It's a temp resolution for SJC customers.

The IPv4 configuration will be offered through DHCP and Cloudinit together.
- IPv4 Method 1: Cloud-Init (Default)
- IPv4 Method 2: DHCP (Offered to the customer for a cloud-init-less solution. )

The IPv6 is only offered by SLAAC;
Since DMIT uses fe80 EUI-64 from the tap device, there is no generic IPv6 gateway address; The guest VM must retrieve the default route through radvd's router announcement.
- IPv6 Method: RADVD, stateless IPv6 assignment.
- Additional benefit: with IPv6 forwarding enabled, and tunnel technology, all the devices in your tunnels could get an IPv6 by RA automatically.
/var/log/DMIT-NOC.log
[SJC] Cloud-init route injection has been completed. Stop and Boot could solve no routing issue.
50% SJC VMs have been move to new network arch. Rest will be done today afternoon PST time.
The DMIT Layer 3 network enhancement has been done in all SJC nodes for a week.

Please check if the network of your VM works as expected. Please feel free to reach us if you've seen any problems.
TOS 3. Account Management

+ 3.9. DMIT does not allow any type of account transfer. DMIT reserves the right to terminate accounts immediately and not to issue refunds.

https://www.dmit.io/pages/tos
Switching Hong Kong CN2 vendor to CTGnet CN2 GIA; couple interruptions are expected.
Direct connection to CTGnet CN2 GIA 100% routing has been completed.
/var/log/DMIT-NOC.log
Direct connection to CTGnet CN2 GIA 100% routing has been completed.
HK -> CN: All thru AS4809(via AS23764)

CN -> HK:
AS4134 -> AS4809(via AS23764);
AS4837 -> AS4809(via AS23764); or AS10099 -> AS23764
AS9808 -> AS58453.
detected FPC error on one line card of Hong Kong Core Router. Emergency reboot FPC.
/var/log/DMIT-NOC.log
detected FPC error on one line card of Hong Kong Core Router. Emergency reboot FPC.
Confirmed hardware failure.
We are ordering new MPC line card;
You might experience decreased service level for all services in Hong Kong recently.
/var/log/DMIT-NOC.log pinned «Confirmed hardware failure. We are ordering new MPC line card; You might experience decreased service level for all services in Hong Kong recently.»
We have an unscheduled emergency maintenance in Hong Kong;
It leads to network downgrade in 30 minutes.

It will be resolved very soon.
/var/log/DMIT-NOC.log
Confirmed hardware failure. We are ordering new MPC line card; You might experience decreased service level for all services in Hong Kong recently.
Withdrawal NTT and Equinix IX routes in temporary; the line card failure is unpredictable right now.

This should lower the impact during line card failure.

The new line card is going to be shipped out from US today via expedited shipping.