We will issue a 7-day equivalent credit to your account.

Please contact us via ticket. In order to process faster, you could add [TYO SLA] to the front of the title. This is optional.

DMIT.io maintenance; ETA 2hr

Extend for 2hr;

The maintenance has complete 2hrs ago;
We've improved the performance of control panel.
Please let us know if you found any error or hard-to-use case in this new control panel.

Thanks for assistance.

DMIT is now support PTR record update;

For IPv6, we also support /64 PTR wildcard record;
But the more specific record for IPv6 will not be supported.

PTR update can be accessed by click the change hostname icon on the right side of your hostname.

The AUP will be also appied to PTR.
DMIT does not support Dest. TCP 25 outbound access.

The snapshot feature has become available in LAX.
Every LAX Pro instance has 1 quota; sPro has 2 quotas.
The extra snapshot upgrade and selection on order will be available soon.

This feature will become available for SJC next, then HKG, last is TYO.

SJC is out of hardware resources.

The new SSD, RAM, and servers are on the way.

1. Two nodes will be rebooted next weekend ( March 17 or 19) for a RAM upgrade.

2. The block storage will be doubled, and the snapshot will be available this weekend (March 10 or 12).

3. Your VM might be cold-migrated (one reboot) without notice to a new node since tight hardware resources.

4. The new instance purchase will be available the next day after the action ( March 11 or 13. ) if the no.2 action is completed on time.

We have to performe emergency reboot for some SJC node. It will be done very soon.

We've noticed IO Error in SJC; Investigating. Keep you posted.

The network components we used implemented hash Layer3+4 for Bond interface, which is not supported by Infiniband.

It caused the disonnection-dead-loop for the entire SJC Ceph cluster.

We've removed the config implemented by component and locked it.

We experience extramly high load in SJC, the new hardware is on the way;
The new NVMe block storage hardware will be installed tomorrow.

We are working on restore Ceph-OSD; The problem is found; It still takes more time to recovery.

We are still working on it; we suggest do not reboot if you still able to run your system, since the I/O is currently suspended.

The remote hand is on the way to the SJC location to implement hardware requirements for repair.

The SLA solution will be posted after repair done.

OSD recovery, backfill in progress.

Step 1 still need ~4hrs; 70% VM will return to normal;

Step 2 will take another ~4hrs; 99% VM will return to normal;

Step 3 needs whole day, it only leads to IO performance impact but not uptime impact.

The SLA is lower than TOS offered. The reimbursement will issue case by case; please submit ticket after the event end.

We are deeply sorry for the recent SLA drop that may have caused inconvenience to your business operations. We understand the importance of our services to your business and we take full responsibility for this interruption.

The fault report will be posted after the event.

Ceph does not allow to run after partitial recovery; step 2 in process.

Step 2 complete;

Due to one OSD failed to be recovery, and data difference during time; there are 13/512 (2.5390625%) data is unable to recovered.

Once again, we apologize for any inconvenience or concern that this may have caused. We value your trust and we will continue to work hard to earn and maintain it.

We regret to inform you that the loss of certain Ceph PG objects may prevent your VM's filesystem from mounting a hard drive during system boot. While this issue can be resolved manually, addressing over 2,000 VMs at this location falls outside the scope of our unmanaged services. However, we would like to offer a compensation package as a token of apology.

====Compensation Package====
DMIT will provide a more detailed fault report, and in the meantime, we will extend your service period by 30 days and double your transfer capacity in your VM permanently. (For UNMETERED plans, we will double your bandwidth as well.)
====Compensation Package====

To address the issue, we will need to take the following steps:

1. DMIT will stop all instances;
2. Client tries to start them one by one;
3. If not able to get into the system:
a) Instance cannot boot: reinstall system, the major(maybe header) object of your drive has permanent loss; rebuild is required.
b) Instance bootable, but system hang: Filesystem failed; manual fix or rebuild;

Please accept our apologies for any inconvenience caused. We appreciate your patience and understanding as we work to address this issue.

After all these, DMIT will first issues 30 days service extension. Then, double the resources. No ticket is required for this.

The fault report will be ready after all these.

[This has been emailed formally]