FTP/rsync server down
Resolved - This incident has been resolved.
Update - We are continuing to investigate this issue.
Monitoring - We have brought the FTP/rsync server back online and are monitoring for issues.
Investigating - The FTP/rsync server is down and we are working on the issue.
Resolved - This incident has been resolved.
Update - We are continuing to investigate this issue.
Monitoring - We have brought the FTP/rsync server back online and are monitoring for issues.
Investigating - The FTP/rsync server is down and we are working on the issue.
Hello everyone. We know there are currently many problems with Myrient. The following is a non-exhaustive list of issues:
- FTP server offline
- rsync server offline
- Increased HTTP 429 errors
- Increased HTTP 500 errors
That is due to the high costs associated with running such a service. For all of Myrient's existence, we have been paying for a large percentage of monthly costs out-of-pocket. The monthly costs are as follows:
- Storage: $3137/month
- Servers: $541/month
- Offsite backups (shared with hShop): $138/month
"Storage" refers to the 432 TB of HDD storage which stores all of Myrient's content, along with 40 Gbps of bandwidth. "Servers" means the FTP and rsync servers, with 10 Gbps of bandwidth shared between them. "Offsite backups" is the cold storage used to store backups of Myrient's database and parts of the site frontend. The monthly costs in total is $3816. The total monthly donations on average was $1300. This means I have been paying the remaining $2516 to keep Myrient online. This is not sustainable for the future of Myrient.
While I appreciate everyone who has donated so far, it is clear that donations are not enough. More will have to be done to decrease costs, such as removing content from the site, decreasing the amount of bandwidth available, and removing certain features, such as FTP and rsync servers. These actions will cause a worse experience. I strive to keep Myrient free of ads and keep download speeds high, but this is becoming increasingly difficult.
This month for the first time in Myrient's history, 100% of the monthly costs was covered by donations. I have ordered more hardware and expect the issues mentioned to be resolved by the end of next week. The future of Myrient is in your hands, so please keep this in mind the next time you are using the service.
- FTP server offline
- rsync server offline
- Increased HTTP 429 errors
- Increased HTTP 500 errors
That is due to the high costs associated with running such a service. For all of Myrient's existence, we have been paying for a large percentage of monthly costs out-of-pocket. The monthly costs are as follows:
- Storage: $3137/month
- Servers: $541/month
- Offsite backups (shared with hShop): $138/month
"Storage" refers to the 432 TB of HDD storage which stores all of Myrient's content, along with 40 Gbps of bandwidth. "Servers" means the FTP and rsync servers, with 10 Gbps of bandwidth shared between them. "Offsite backups" is the cold storage used to store backups of Myrient's database and parts of the site frontend. The monthly costs in total is $3816. The total monthly donations on average was $1300. This means I have been paying the remaining $2516 to keep Myrient online. This is not sustainable for the future of Myrient.
While I appreciate everyone who has donated so far, it is clear that donations are not enough. More will have to be done to decrease costs, such as removing content from the site, decreasing the amount of bandwidth available, and removing certain features, such as FTP and rsync servers. These actions will cause a worse experience. I strive to keep Myrient free of ads and keep download speeds high, but this is becoming increasingly difficult.
This month for the first time in Myrient's history, 100% of the monthly costs was covered by donations. I have ordered more hardware and expect the issues mentioned to be resolved by the end of next week. The future of Myrient is in your hands, so please keep this in mind the next time you are using the service.
The new hardware has been installed and is now live. The HTTP 429 errors (rate limits) have been removed and HTTP 500 errors have been eliminated. Additionally, the FTP and rsync servers are back online. I will continue monitoring the performance and stability of the infrastructure over the next few days.
After the last announcement, we received an outpouring of support including more frequent donations. Thank you to everyone for still believing in us bouncing back from the issues and downtime.
I would like to address the most frequent concern many have asked about, which is the removal of content. We do not have plans to remove any content at this time. Nobody likes bad surprises, so if we ever have to remove content or make any other changes to Myrient that negatively impacts the user experience, we will give advance notice before doing it. Of course, any content that is removed will still be available in our offsite backup. Once again, thank you for your continued support!
After the last announcement, we received an outpouring of support including more frequent donations. Thank you to everyone for still believing in us bouncing back from the issues and downtime.
I would like to address the most frequent concern many have asked about, which is the removal of content. We do not have plans to remove any content at this time. Nobody likes bad surprises, so if we ever have to remove content or make any other changes to Myrient that negatively impacts the user experience, we will give advance notice before doing it. Of course, any content that is removed will still be available in our offsite backup. Once again, thank you for your continued support!
The FTP and rsync servers are going offline for a short while for maintenance. We will update this announcement as soon as the maintenance is completed.
Resolved - This incident has been resolved.
Update - The FTP and rsync servers are back up and we expect no further issues.
Update - FTP and rsync servers are down again. We're continuing to investigate the issue.
Update - The FTP and rsync servers are back online while we monitor the performance.
Update - We're deploying new hardware at this time.
Resolved - This incident has been resolved.
Update - The FTP and rsync servers are back up and we expect no further issues.
Update - FTP and rsync servers are down again. We're continuing to investigate the issue.
Update - The FTP and rsync servers are back online while we monitor the performance.
Update - We're deploying new hardware at this time.
As a video game preservation service, Myrient always had the policy of not offering content for current generation systems. Over time, however, we've received many requests from everyone to host Nintendo Switch sets on Myrient even though it went against our policy. Due to the high demand, we've decided to make an exception for the Nintendo Switch system.
Starting today, you'll be able to download content for Nintendo Switch from our website. This includes cartridge dumps, eShop releases, updates, DLC, and demos. You can find the sets in "/No-Intro/Nintendo - Nintendo Switch", "/No-Intro/Nintendo - Nintendo Switch (Digital)", "/No-Intro/Nintendo - Nintendo Switch (Digital) (Dev ROMs)", and "/No-Intro/Nintendo - Nintendo Switch (Digital) (DLC)".
Nintendo Switch sets will be available for download on the FTP and rsync servers at a later date.
Starting today, you'll be able to download content for Nintendo Switch from our website. This includes cartridge dumps, eShop releases, updates, DLC, and demos. You can find the sets in "/No-Intro/Nintendo - Nintendo Switch", "/No-Intro/Nintendo - Nintendo Switch (Digital)", "/No-Intro/Nintendo - Nintendo Switch (Digital) (Dev ROMs)", and "/No-Intro/Nintendo - Nintendo Switch (Digital) (DLC)".
Nintendo Switch sets will be available for download on the FTP and rsync servers at a later date.
Network Performance Issues
Resolved - This incident has been resolved.
Monitoring - A fix has been implemented and we are monitoring the results.
Update - We continue to work on the problem of slow download speeds from the website, FTP server, and rsync server.
Monitoring - A fix for slow download speeds on our website has been implemented and we are monitoring the results.
Update - Traffic to our website has been rerouted while we implement a fix.
Update - Download speeds on the FTP and rsync servers have returned to normal. We have started working on implementing a fix for slow download speeds on our website.
Identified - The issue has been identified and a fix is being implemented.
Investigating - We're investigating network performance issues between our backend and caching servers causing extremely slow download speeds.
Resolved - This incident has been resolved.
Monitoring - A fix has been implemented and we are monitoring the results.
Update - We continue to work on the problem of slow download speeds from the website, FTP server, and rsync server.
Monitoring - A fix for slow download speeds on our website has been implemented and we are monitoring the results.
Update - Traffic to our website has been rerouted while we implement a fix.
Update - Download speeds on the FTP and rsync servers have returned to normal. We have started working on implementing a fix for slow download speeds on our website.
Identified - The issue has been identified and a fix is being implemented.
Investigating - We're investigating network performance issues between our backend and caching servers causing extremely slow download speeds.
Increased HTTP 502 Errors
Resolved - This incident has been resolved.
Monitoring - A fix has been implemented and we are monitoring the results.
Identified - We have identified an increased rate of HTTP 502 errors. We are working to resolve the problem.
Resolved - This incident has been resolved.
Monitoring - A fix has been implemented and we are monitoring the results.
Identified - We have identified an increased rate of HTTP 502 errors. We are working to resolve the problem.
During the next few days, we'll be migrating Myrient to a new storage cluster in order to reduce costs by a third, increase reliability, and improve performance. As a result, users may experience slower download speeds than usual until the migration is complete. Thank you for understanding.
Update - The maintenance has been completed.
Update - The maintenance has been completed.
Performance Issues
Due to recent events, Myrient is receiving a massive influx of traffic and all of our caching servers are currently overloaded. Therefore, download speeds are slower than usual. We are looking at potential solutions to the problem with the funding currently available.
Due to recent events, Myrient is receiving a massive influx of traffic and all of our caching servers are currently overloaded. Therefore, download speeds are slower than usual. We are looking at potential solutions to the problem with the funding currently available.
The webserver crashed a few hours ago. We've restarted it and the website is back up.
Performance Issues - Update
We've implemented aggressive CDN caching and increased our caching server capacity to alleviate performance issues for downloads from the website. All content cached (downloaded at least once) should no longer download slowly.
We've implemented aggressive CDN caching and increased our caching server capacity to alleviate performance issues for downloads from the website. All content cached (downloaded at least once) should no longer download slowly.
Myrient's FTP and rsync servers are being phased out
While the ability to download via FTP and rsync is a defining feature of Myrient, the high cost of bandwidth and time spent on server maintenance have made it difficult to support those protocols. Additionally, recent events have forced us to make changes for the security of the service and longevity of all the content on Myrient.
We would like to inform you that Myrient's FTP and rsync servers will shut down on 26 May.
Downloading via the website using tools like wget will continue to be supported and we're continuing to improve upon various strategies, such as caching, to serve content as fast as possible with the funding currently available. We hope this won't be an inconvenience and thank you for understanding.
While the ability to download via FTP and rsync is a defining feature of Myrient, the high cost of bandwidth and time spent on server maintenance have made it difficult to support those protocols. Additionally, recent events have forced us to make changes for the security of the service and longevity of all the content on Myrient.
We would like to inform you that Myrient's FTP and rsync servers will shut down on 26 May.
Downloading via the website using tools like wget will continue to be supported and we're continuing to improve upon various strategies, such as caching, to serve content as fast as possible with the funding currently available. We hope this won't be an inconvenience and thank you for understanding.
Performance Issues - Resolved
We added more caching servers yesterday to further alleviate performance issues for downloads from the website. All content, not just cached files, should no longer download slowly. We will continue to monitor for performance issues related to download speed.
We're currently investigating an issue where downloads are stopping randomly and showing an error.
We added more caching servers yesterday to further alleviate performance issues for downloads from the website. All content, not just cached files, should no longer download slowly. We will continue to monitor for performance issues related to download speed.
We're currently investigating an issue where downloads are stopping randomly and showing an error.
Download Issues - Update
We know there are issues with downloads at the moment. This is due to failures of multiple drives. We're waiting for replacements and as soon as they're received we'll fix the issues as soon as possible.
We know there are issues with downloads at the moment. This is due to failures of multiple drives. We're waiting for replacements and as soon as they're received we'll fix the issues as soon as possible.
Download Issues - Update
We've received and installed replacement drives and the storage cluster is currently rebuilding. Until this is complete, the cluster will continue operating in a degraded state. We would like to emphasize that due to the redundancy of our system, no data was lost.
We've received and installed replacement drives and the storage cluster is currently rebuilding. Until this is complete, the cluster will continue operating in a degraded state. We would like to emphasize that due to the redundancy of our system, no data was lost.
Download Issues - Update
We would like to inform you that at the time of this announcement, all collections and sets are operational again except for the following:
Redump set "Sony - PlayStation 3"
Redump set "Sony - PlayStation 3 - Disc Keys"
Redump set "Sony - PlayStation 3 - Disc Keys TXT"
Until the storage cluster is rebuilt, users downloading content of the sets listed above may continue to encounter issues. We'll let you know once the rebuilding process is completed.
Myrient is experiencing performance issues due to a recent massive increase of traffic. We're provisioning additional servers to resolve the problem. Thank you for your patience.
We would like to inform you that at the time of this announcement, all collections and sets are operational again except for the following:
Redump set "Sony - PlayStation 3"
Redump set "Sony - PlayStation 3 - Disc Keys"
Redump set "Sony - PlayStation 3 - Disc Keys TXT"
Until the storage cluster is rebuilt, users downloading content of the sets listed above may continue to encounter issues. We'll let you know once the rebuilding process is completed.
Myrient is experiencing performance issues due to a recent massive increase of traffic. We're provisioning additional servers to resolve the problem. Thank you for your patience.
Download Issues - Update
At the time of this announcement, the Redump sets "Sony - PlayStation 3", "Sony - PlayStation 3 - Disc Keys", and "Sony - PlayStation 3 - Disc Keys TXT" are operational again and the process of rebuilding the storage cluster has been completed. Additionally, the issue where downloads are stopping randomly and showing an error has been resolved.
We're still provisioning additional servers to improve slower download speeds than usual for certain content. Thank you for your continued patience.
At the time of this announcement, the Redump sets "Sony - PlayStation 3", "Sony - PlayStation 3 - Disc Keys", and "Sony - PlayStation 3 - Disc Keys TXT" are operational again and the process of rebuilding the storage cluster has been completed. Additionally, the issue where downloads are stopping randomly and showing an error has been resolved.
We're still provisioning additional servers to improve slower download speeds than usual for certain content. Thank you for your continued patience.
Scheduled Maintenance
We'll be performing scheduled maintenance on a select number of servers from 12 July at 22:00 UTC to 13 July at 22:00 UTC. During this time, the following content may be temporarily unavailable:
All Miscellaneous sets
All TOSEC sets
All TOSEC-ISO sets
All TOSEC-PIX sets
All other content will continue to be available and we'll provide updates as necessary. Thank you for understanding.
Update - The scheduled maintenance has been extended to 14 July at 17:00 UTC.
Update - The scheduled maintenance has concluded.
We'll be performing scheduled maintenance on a select number of servers from 12 July at 22:00 UTC to 13 July at 22:00 UTC. During this time, the following content may be temporarily unavailable:
All Miscellaneous sets
All TOSEC sets
All TOSEC-ISO sets
All TOSEC-PIX sets
All other content will continue to be available and we'll provide updates as necessary. Thank you for understanding.
Update - The scheduled maintenance has been extended to 14 July at 17:00 UTC.
Update - The scheduled maintenance has concluded.
Scheduled Maintenance
We'll be performing scheduled maintenance on a select number of servers from 14 July at 00:00 UTC to 14 July at 17:00 UTC. During this time, the following content may be temporarily unavailable:
No-Intro set "Microsoft - MSX"
No-Intro set "Microsoft - MSX2"
No-Intro set "Microsoft - Xbox (Development Kit Hard Drives)"
No-Intro set "Microsoft - Xbox 360 (Development Kit Hard Drives)"
No-Intro set "Microsoft - Xbox 360 (Digital)"
All other content will continue to be available and we'll provide updates as necessary. Thank you for understanding.
Update - The scheduled maintenance has concluded.
We'll be performing scheduled maintenance on a select number of servers from 14 July at 00:00 UTC to 14 July at 17:00 UTC. During this time, the following content may be temporarily unavailable:
No-Intro set "Microsoft - MSX"
No-Intro set "Microsoft - MSX2"
No-Intro set "Microsoft - Xbox (Development Kit Hard Drives)"
No-Intro set "Microsoft - Xbox 360 (Development Kit Hard Drives)"
No-Intro set "Microsoft - Xbox 360 (Digital)"
All other content will continue to be available and we'll provide updates as necessary. Thank you for understanding.
Update - The scheduled maintenance has concluded.
Increased HTTP 403 errors
We're investigating an increase in HTTP 403 errors when downloading the following content with Rclone and various other clients:
No-Intro set "Microsoft - MSX"
No-Intro set "Microsoft - MSX2"
No-Intro set "Microsoft - Xbox (Development Kit Hard Drives)"
No-Intro set "Microsoft - Xbox 360 (Development Kit Hard Drives)"
No-Intro set "Microsoft - Xbox 360 (Digital)"
All other content is unaffected and we'll provide updates as necessary. Thank you for understanding.
Update - A fix has been implemented and we're monitoring the results.
Update - This incident has been resolved.
We're investigating an increase in HTTP 403 errors when downloading the following content with Rclone and various other clients:
No-Intro set "Microsoft - MSX"
No-Intro set "Microsoft - MSX2"
No-Intro set "Microsoft - Xbox (Development Kit Hard Drives)"
No-Intro set "Microsoft - Xbox 360 (Development Kit Hard Drives)"
No-Intro set "Microsoft - Xbox 360 (Digital)"
All other content is unaffected and we'll provide updates as necessary. Thank you for understanding.
Update - A fix has been implemented and we're monitoring the results.
Update - This incident has been resolved.