Ongoing, you will receive opened ticket and email with new IPs for each instance that include 205.210.164.0/22.

Please check your ticket first for any issues, please note the 2nd IP will become primary IP; If you have 2+ IPs, please use the 2nd IP to log into your VM.

All jobs are done.

205.210.164.0/22 has been returned to our supplier.

DMIT@re1.lax.DMIT.IO> show route 205.210.164.0

inet.0: 783306 destinations, 1860134 routes (783298 active, 8 holddown, 1 hidden)
+ = Active Route, - = Last Active, * = Both

0.0.0.0/0 *[Static/5] 13w1d 02:26:48
> to ****** via xe-3/0/2.0

GIA.inet.0: 789749 destinations, 1913818 routes (789741 active, 8 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

0.0.0.0/0 *[Static/5] 13w1d 02:26:48
> to ****** via xe-3/0/2.0

We received dozens of DMCA abuse taken down requests recently. We have to let you know any company and datacenter in the United States have to follow the regulations and laws of the US.

Please do not download and deliver the content you have no copyright. There are many honeypot servers which run by the owner of the copyright and US GOV. They will detect illegal operations if you are downloading some pirated resources.

According to our TOS and AUP, DMIT Inc is a United States company; we do have many upstream and cooperates who located in the US. The LAX data center also located in the US. If we do not take down your server when we received DCMA takedown notice, we might be prosecuted, and the equipment in the LAX data center might be taken. Our upstream might shut down our connection. So, please do follow our AUP and TOS.

Once we received the DMCA, taken down requests. For each server, for the first time, we will suspend it for seven days, or you pay 25USD fine; you have to pay 25USD after the first time. You have to reinstall the system if you cannot prove you have the copyright; you have to take action to prevent these things happen again; we keep right to refuse services if we consistently receive a DMCA takedown notice on your account.

We will share our abuse-mailbox via the website publicly soon.

So, you can check the abuse history via our website.

But the takedown review and operations (suspend) still on manually.

Start at Feb 19, 2020; there are some customers report packet loss and offline issues. We already find the problem and it will be fixed soon. ETA before Feb 25, 2020 EST. We will issue 5 day billing cycle and 10% traffic waived.

-NOC

New nodes will be ready within today (EST Feb 25). We will start to migrate VM by tickets time order.

We will put twice the length of downtime (EST Feb 19, 2020 to VM successfully migrated from 19th) with the corresponding Credit to your DMIT account. Also the 10% traffic will be waived later.

Ready for migration, please do not update your ticket, we will handle it by time reverse order;

uBuilder, Builder, and Anti-DDoS users have priority.

All the guest VM will finally be migrated to brand new nodes; Cascade Lake / 2nd EPYC; Randomly;

As our investigation, all these problems caused by our architecture designed and Linux Kernel issues.

For resource control reason;
We will update package configuration recently;
The current order will not be affected.

The low-costs product will no longer be provided to save hardware resources and client support resources. If you have it, keep it.

Tips: If you got the product that we no longer provided. Credit Card payment will be the best one to avoid over-due payment; PayPal always charges the client later than the invoice payment date.

The price will be increased on PVM.LAX.Pro.u Bandwidth Unmerited Series server.

http://ping.pe/2.56.184.2
Demo, after migration.

NOC & Support :
90% of tickets have been completed. If you have packet loss and occasionally offline issues, please contact us via tickets; EPYC nodes will arrive our datacenter March 10+, 2020; After that, all customer will be migrated to EPYC node.

Sales :
1. Due to the overall price increase of CN2 products by first-class suppliers(The IDC which has CN2 connection directly) on the west coast of the United States; We will adjust the price according to market rules soon. Also, another reason is we have to save bandwidth for our full price IP Transit and Dedicated Bandwidth customers, since China Telecom will not approve new CN2 GIA contract and orders now.

2. Our 2nd Anniversary Entity Customization Gift (customized Macbook Pro & Macbook Air) has reached our warehouse; We will ship it to our customers through DHL within 7 days.

NOC:
We will enable L7 and always-on DDoS mitigation for our Anti-DDoS network configuration on LAX. Please note, except China, all the ICMP will be dropped in defaults. The speed test will become incorrect.

Sales:

The packages of PVM.LAX.Pro are updated. The original price you got will not be changed.

Please note, please do not actively renew your VM, which means renew your VM by "renew now" function on the control panel. This will lead to the promotion code and/or original price you have lost.

All the VM on the new order will be deployed on our new nodes (2nd Gen EPYC & Cascade). The VM from history order will be migrated later once we have enough nodes.

P.S.: Sorry for the incomplete message, due to cost control, The VM in the history orders might be migrated into Skylake or Kaby Lake nodes.

NOC:

L7 and Always-On already enable; the L7 protection will only be able to use in HTTP protocol; HTTPS needs a certificate and offload. We will not offer that now. BTW,

Please do not use it as a proxy; we will not help the customer find and solve the issues if they use the Anti-DDoS network as a proxy; because the L7 and Always-on will lead to some unknown fault on proxy services.

The ICMP from everywhere except China has been dropped.

LAX packet loss compensation has been all distributed; The 2y anniversary gifts have all been ship out.

Due to the impact of the epidemic situation; Remote maintenance, transportation and ISP maintenance of our data center in Los Angeles are all affected.

The following NOC matters have been postponed:

- We planned to connect with Zayo (AS6461) in March, 2020 to improve the quality of local connections in the United States. [ No further ETA from Zayo IPT delivery team ]

- New EPYC node RAM: We ordered a dozen sets DDR4 memory from our supplier; they ran out of their stock; [ No ETA for memory shipment ]

- New EPYC node: 2 sets of the server already shipped; ETA arrival is Mar 22, 2020; [ New ETA: March 30, 2020 ]. There are still have few sets not been shipped yet; [ No ETA ]

- Enhanced Backbone quality and SLA: We do have only 1 unprotected backbone in our LAX Extended PoP and LAX Datacenter; We already ask for upgrade it to protected loop at March 12, 2020; [ But there is no ETA for this ]

- We found that several Level 1 errors are reported from the transceiver every day; Our NOC expects that due to unpredictable fiber quality degradation and transceiver laser aging (the laser quality is expected to be lower than the lowest RX threshold of the laser within 1-3 months). We will add amplifiers at both ends of the transceiver and replace it with a new transceiver. At the same time, we will use DWDM to enhance reliability. All these tasks need to be synchronized doing after the ETA is obtained in the previous task to reduce the down time.

The issues on VM Control Panel have been fixed. We have to reboot Node 27J3B42 and 2VC8Q22 on this. There is no impact on other nodes and VM.

We detected Huge TCP DDoS Attack to our client on LH36806/7 and gateway; we have to reboot LH36806 to recover the dead kernel status. We already enhanced our Anti-DDoS rule.

Due to a large number of orders again, our two Xeon Gold nodes were severely overloaded. (20% more than our design). At the same time, due to DDoS attacks and high load, the node continuously disconnects from the cluster. This caused the task unable to execute; the continuous disconnection of the cluster eventually lead to the accumulation of tasks and the failure of the cluster.

The 2 HPE EPYC nodes we ordered from the supplier have arrived at the data center at present; the memory is still waiting for shipment; we cannot be used in the production environment for the time. Since no enough RAM.

We have ordered one more DL360 node directly from HPE. The DL360 Xeon Gold node will be ready in 5-7 days. it's the same with our current Xeon gold nodes.

We are very sorry that we failed to anticipate the expansion of our business far beyond our imagination.

We may stop accepting new orders from LAX in the near future. In order to ensure that the current service can basically fun safely.

The reason we talk before is not exact correct. After in-depth investigation; Due to defects in PVE-Firewall. DMIT recompiled it; We delete defective designs; This also led to a failure in filtering broadcast and multicast packets successfully. We have captured a large number (> 100kpps) of broadcast or multicast packets sent by a guest VM when the network rate decreases and packet loss increases. The kernel of the guest VM does not have enough buffer and performance to process these multicast packets, which causes congestion in the guest VM. Since multicast packets are distributed to each guest VM; When the guest VM has a high load, it also causes a sudden high load of Host; (Since this is a shared VM, each VM does not have a full CPU core).

P.S: Some special intranet broadcast packets are easily blocked in a large-scale network environment. There are some packets that will cause network architecture changes (e.g. advertise their IP as IGMP Snooping router)

Our engineer team already put a new beta optimized PVE-Firewall to the nodes that have these issues for testing. During our test, it succeeded in blocking intranet attacks and abuse. Our engineer will upload the PVE-Firewall to our code platform and deploy it to all nodes.

However, our number of nodes still exceeds our expected number and we will not accept new orders for the time being. (Upgrade is allowed). Although the node resources become sufficient due to VM being released recently; new orders will be accepted when the new node is ready.

Scheduled maintenance:
3PM PST
Apr 7, 2020.
Window: 30min~1h

Maintenance:
1. Check the X-C quality,
2. Check light attenuation of our backbone.
3. Replace the transceiver on both side of backbone

Temporary Impact:
1. The network LH36806/7 will be inaccessible.
2. IX Peering goes down.
3. Network capability goes down.