Libcast.com Infra status

February 2017

No incidents reported for this month.

January 2017

No incidents reported for this month.

December 2016

Transcoding may be slow due to OVH Public Cloud issue
Issue resolved.
Dec 1, 17:23 - Dec 20, 18:32 CET

November 2016

Libcast Maintenance - November 29th at 06am
The maintenance operation is now finished. Thanks you for your patience.
Nov 22, 11:00 - Nov 29, 07:12 CET
Issue with OVH IPLB [this concernes only a few customers]
Issue has been resolved. Still, we encourage our customers to update their DNS as advised here : https://support.libcast.com/hc/fr/articles/203234532-Comment-personnaliser-mon-projet-avec-mon-nom-de-domaine
Nov 23, 10:54-17:41 CET

October 2016

No incidents reported for this month.

September 2016

Hardware issue on our OVH DedicatedCloud
All services are back and running. We are extremely sorry for the inconvenience. We will write a "post mortem" here as soon as we have more information on what happened.
Sep 23, 17:19-18:17 CET
Issue with API uploads
An error from yesterday update maid the API endpoint's root (https://api.libcast.com/) unavailable for some users. We have fixed this issue. The problem was not directly related to upload, it was limited to the root. Everything is back to normal now. We are extremely sorry for the inconvenience.
Sep 22, 11:18-14:05 CET

August 2016

Problem with OVH NAS HA
A NAS HA volume has been malfunctioning this morning making some backend pages unavailable. This issue has been resolved and the service is fully operational now.
Aug 8, 09:56 CET

July 2016

Analytics update
Stats are now displayed in real-time again. Some accounts may see cached analytic data for a few more hours, which is normal.
Jul 4, 18:37 - Jul 5, 18:52 CET
Problem with OVH vRack technology and some NAS HA volumes
This incident has been resolved.
Jul 1, 08:16-11:38 CET

June 2016

Issue connecting our ACL server
The cluster has been fully synchronized. Service is back and should remain fully operational.
Jun 24, 17:30-17:38 CET
Issue connecting our ACL server
Issue is fixed. We will provide more information in a post-mortem message here.
Jun 24, 17:19-17:22 CET
Incident with an OVH NAS HA volume
One of the High Availability NAS volume provided by our ISP OVH has been disconnected (http://travaux.ovh.net/?do=details&id=18532&PHPSESSID=5efad51e201ae6c507939cef90f4dfdc). This caused our application to answer almost all requests with an "Oops" error message. We re-mounted the volume this morning and the issue is now fixed. We are sorry for the inconvenience.
Jun 10, 09:04 CET

May 2016

Minor issue on transcoding servers
This incident has been resolved.
May 26, 16:22-17:15 CET
Unidentified incident with our permission server
The service is fully operational. We are extremely sorry for the inconvenience.
May 17, 17:37-18:03 CET
Analytics migration
Stats are now real-time! Migration is over and the new system is fully operational!
May 10, 11:46 - May 15, 17:27 CET

April 2016

Customers with a dedicated domain name may encounter an issue
OVH IP Load Balancing is up again. Sorry for the inconvenience.
Apr 18, 16:31-18:37 CET
[Scheduled] Operation on Libcast cloud infrastructure
Maintenance is over!!
Apr 1, 00:12-02:04 CET

March 2016

Incident with our ACL server
ACL server has been rebooted and everything is back to normal!
Mar 16, 13:49-13:58 CET

February 2016

Incident caused by OVH storage infrastructure
An issue regarding OVH storage infrastructure caused a minor outage on our infrastructure. More details here: http://travaux.ovh.net/?do=details&id=16809 Everything is back to normal.
Feb 28, 22:14 CET
Issue with file upload
Some uploading issues occurred during an operation regarding our storage infrastructure. Everything is working fine again. Sorry for the inconvenience.
Feb 17, 18:05 CET
Incident on webservices
An issue with a tiers application caused degraded performances on Libcast webservices early in the morning. This issue already occurred last week and we are working hard on a permanent fix. We should be able to prevent this from ever happening again in the next Libcast update at the end of the week.
Feb 10, 09:50 CET
Minor incident on webservices
Webservices observed a ~10min downtime due to a tiers application slowdown. Everything is back in order now.
Feb 3, 22:15 CET

January 2016

Major issue with our DedicatedCloud infrastructure
The service is back now. Everything is working well. It seems the issue was related to a network mis-configuration. We are extremely sorry for the inconvenience
Jan 14, 11:39-11:58 CET
Slow transcoding due to OVH API errors
OVH API still slow, but no more error. We increased our timeouts and are able to provision transcoder servers again! We plan to deploy more static servers in the near future. Sorry for the inconvenience.
Jan 12, 17:54 - Jan 13, 13:11 CET
Libcast Webservices are down
Problematic requests have been detected. We made sure the webservices are up and running again! Sorry for the inconvenience.
Jan 9, 13:58-14:43 CET
PostgreSQL over-charge
An incident with our relational databases caused connection timeouts from front servers immediately followed by a global outage. The incident is now closed. Sorry for the inconvenience.
Jan 8, 14:43 CET

December 2015

Slow transcoding due to OVH API errors
Due to errors coming from the OVH API we have not been able to provision extra servers to handle transcoding high demand. Those errors have been fixed and the transcoding is going faster now.
Dec 1, 15:00 CET

November 2015

Analytics temporarly unavailable
Past days analytics is still being processed and everything is getting back to normal
Nov 17, 11:15 - Nov 22, 13:22 CET
Redis server crashes
This incident has been resolved.
Nov 21, 12:33-12:34 CET
Infra update
This incident has been resolved.
Nov 4, 00:12-08:34 CET

October 2015

Webservice incident
An issue with our analytics database caused Libcast webservices to be down for up to 15min. This issue is resolved now. We are sorry for the inconvenience.
Oct 30, 18:06 CET
Incident with OVH PublicCloud API
This incident has been resolved.
Oct 21, 18:06-21:08 CET
Outage with PostgreSQL Fail Over IP
This incident has been resolved.
Oct 16, 12:05-12:08 CET
[Scheduled] PostgreSQL Fail Over test
The scheduled maintenance has been completed.
Oct 13, 23:00 - Oct 14, 00:00 CET
[Scheduled] LB Cluster Maintenance
The scheduled maintenance has been completed.
Oct 9, 00:00-01:00 CET
Problem with OVH NAS / HA Storage volumes
Resolved by OVH
Oct 2, 18:11-18:13 CET

September 2015

OVH IP Load Balancers are down
IP Load Balancers are back in business! We are extremely sorry for the inconvenience.
Sep 30, 16:26-16:39 CET
Service down
ACL server is back!
Sep 29, 13:41-14:17 CET
[Scheduled] DB Maintenance
The scheduled maintenance has been completed.
Sep 28, 23:00 - Sep 29, 00:30 CET
[Scheduled] Libcast update
The scheduled maintenance has been completed.
Sep 14, 23:15 - Sep 15, 00:30 CET
Transcoding paused
Transcoding has been unpaused despite of the fact we still have minor issues with our filesystem.
Sep 9, 12:42-20:30 CET
System crash on ACL server
A major hardware issue made the ACL server crash.
Sep 3, 23:30 CET