We are cleaning the database, it working on its maximum throughput. The situation will stabilize soon.
You may expect small delays working with telematics messages database.
Sorry for any inconveniences.
You may expect small delays working with telematics messages database.
Sorry for any inconveniences.
downtime started, error: failed to connect client to MQTT broker
---
downtime ended, period: 12 seconds
---
downtime ended, period: 12 seconds
We've detected a bug in the MQTT broker, which is being hot-fixed in the moment. Sessions that caused this misbehaviour are temporary blackholed.
The bug with MQTT Broker was fixed, all related sessions that initiated the incindent were resumed.
Sorry for any inconveniences!
Sorry for any inconveniences!
Dear flespi user!
If you are not using flespi analytics for your services please ignore message below.
We are now moving analytics calculation services to new hardware servers. While these services are still running in experimental mode there is no uninterruptable redundancy service provided.
While we move services to new servers, one by one, the calculation cache should be initialized and synchronized for each caluclated device before the calculation. As we have now more then 150.000 devices and more than 270.000 calculators in the platform this process is estimated to 1 hour.
It means that some calculators may generate events and calculate devices with the delay, up to 1 hour in the worsest case. You can monitor the status per each specific calculated device by it 'synced' property value.
If you are not using flespi analytics for your services please ignore message below.
We are now moving analytics calculation services to new hardware servers. While these services are still running in experimental mode there is no uninterruptable redundancy service provided.
While we move services to new servers, one by one, the calculation cache should be initialized and synchronized for each caluclated device before the calculation. As we have now more then 150.000 devices and more than 270.000 calculators in the platform this process is estimated to 1 hour.
It means that some calculators may generate events and calculate devices with the delay, up to 1 hour in the worsest case. You can monitor the status per each specific calculated device by it 'synced' property value.
Dear flespi users!
Starting from the 07:00 UTC our uplink provider will do some changes in the network routing scheme at the data center hosting the flespi platform. We do not expect any impact to the platform more than a few short disconnects.
If you have any questions, please contact us in the chat embedded in the flespi.io panel or on forum.flespi.com
Starting from the 07:00 UTC our uplink provider will do some changes in the network routing scheme at the data center hosting the flespi platform. We do not expect any impact to the platform more than a few short disconnects.
If you have any questions, please contact us in the chat embedded in the flespi.io panel or on forum.flespi.com
Dear flespi users!
Next Monday, on October 14, we will install an update with the small change in our Telematics Gateway REST API.
Once installed, the devices creation method (POST, https://flespi.io/docs/#/gw/!/devices/post_devices ) will require the "device_type_id" and "configuration" fields to be specified. Also, the "configuration" object for all device types will require an "ident" field.
Please review your integrations and correct them to always include such fields when you're creating devices.
If you have any questions, please contact us in the chat embedded in the flespi.io panel or on forum.flespi.com
Next Monday, on October 14, we will install an update with the small change in our Telematics Gateway REST API.
Once installed, the devices creation method (POST, https://flespi.io/docs/#/gw/!/devices/post_devices ) will require the "device_type_id" and "configuration" fields to be specified. Also, the "configuration" object for all device types will require an "ident" field.
Please review your integrations and correct them to always include such fields when you're creating devices.
If you have any questions, please contact us in the chat embedded in the flespi.io panel or on forum.flespi.com
Dear flespi users!
Today, starting 07:00 UTC we will update our primary router and firewall with new filtering and routing platform. Internally we are changing iptables based rules to nftables and this is what affect the whole network packets filtering and routing process of flespi platform.
Newly developed nftables based routing platform was tested in few locations and environments, but as it happens sometimes something can go wrong. After the upgrade we will check it carefully and revert back in case any problem is detected immediatelly.
We expect connection flushes and short unavailability of our services to Internet for a few seconds during platform replacement.
We will post you the result of upgrade later. Thank you for understanding!
Today, starting 07:00 UTC we will update our primary router and firewall with new filtering and routing platform. Internally we are changing iptables based rules to nftables and this is what affect the whole network packets filtering and routing process of flespi platform.
Newly developed nftables based routing platform was tested in few locations and environments, but as it happens sometimes something can go wrong. After the upgrade we will check it carefully and revert back in case any problem is detected immediatelly.
We expect connection flushes and short unavailability of our services to Internet for a few seconds during platform replacement.
We will post you the result of upgrade later. Thank you for understanding!
The firewall & routing platform was upgraded to the new version. During upgrade process there were no problems at all for the established connections and all external traffic (Internet<=>flespi) flowed as usual. We only experienced some connection failures (with three quarters of connection attemps failed) trying to access flespi external IP adresses (e.g. flespi.io or mqtt.flespi.io) from inside of our network. But once connection was established the traffic through it flowed as usual. So basically only subsystems like REST API via MQTT or telematics streams from flespi to flespi were affected.
Everything was fixed approximately 1 hour ago and we do not see any kind of anomalities anymore.
Everything was fixed approximately 1 hour ago and we do not see any kind of anomalities anymore.
Dear flespi users!
We will install today additional automatic fail-over system with connection state replication on our primary gateway servers. This will result in few connections flushes to flespi services plus may be few seconds of IP addresses unavailability during ARP cache invalidation on all connected servers. In worse case we may even have short downtime. Anyway, please be aware about the operations today with flespi gateway servers.
As a result, after everything is installed and configured we end up with fully automated gateway services based on very ancient ‘heartbeat’ package which development stopped in 2012, but it is still absolutely stable as Kalashnikov machine gun. Initially we started automated gateway services 3 years ago with its successor - ‘crm’, but its behavior was quite uncontrolled and after some problems we switched back to manual hand-mode. Now it is again time for automation, but based on the different system.
And, of course, wish a good Friday!
We will install today additional automatic fail-over system with connection state replication on our primary gateway servers. This will result in few connections flushes to flespi services plus may be few seconds of IP addresses unavailability during ARP cache invalidation on all connected servers. In worse case we may even have short downtime. Anyway, please be aware about the operations today with flespi gateway servers.
As a result, after everything is installed and configured we end up with fully automated gateway services based on very ancient ‘heartbeat’ package which development stopped in 2012, but it is still absolutely stable as Kalashnikov machine gun. Initially we started automated gateway services 3 years ago with its successor - ‘crm’, but its behavior was quite uncontrolled and after some problems we switched back to manual hand-mode. Now it is again time for automation, but based on the different system.
And, of course, wish a good Friday!
Dear flespi users!
Due to upgrade issues automatic analytics system is currently out of working state and no events will be generated into MQTT. We expect repair time in next 10-30 minutes plus additional time (up to 60 minutes) will take to sync all calculators with their state. Sorry for any inconveniences.
Due to upgrade issues automatic analytics system is currently out of working state and no events will be generated into MQTT. We expect repair time in next 10-30 minutes plus additional time (up to 60 minutes) will take to sync all calculators with their state. Sorry for any inconveniences.
Automatic analytics system is up and is being synchronizing messages from databases at the moment, events processing for most of the devices will follow soon.
Dear flespi users!
We have prepared a minor change regarding HTTP streams API.
In general we will support a new format of the headers field for HTTP stream.
Both formats will exist simultaneously until November 18, then the old format will become unavailable.
More details here:
https://forum.flespi.com/d/25-changelog-flespi-telematics-hub/14
We have prepared a minor change regarding HTTP streams API.
In general we will support a new format of the headers field for HTTP stream.
Both formats will exist simultaneously until November 18, then the old format will become unavailable.
More details here:
https://forum.flespi.com/d/25-changelog-flespi-telematics-hub/14
flespi forum
[Changelog] flespi telematics hub
Place for information about telematics hub updates.
Dear flespi users!
Today we will perform planned servers OS maintenance. This is a controlled process and we do not expect any impact on the platform services, but you may experience short reconnects to MQTT broker while we upgrade this or that server.
Just for your information. Have a good & peaceful day!
Today we will perform planned servers OS maintenance. This is a controlled process and we do not expect any impact on the platform services, but you may experience short reconnects to MQTT broker while we upgrade this or that server.
Just for your information. Have a good & peaceful day!
downtime started, error: Failed to perform REST API call that performs meta-data modifications. This usually indicates that all meta-data database operations are unavailable.
We've faced an issue with the automatic failover subsystem for the metadata database. Working on it now.