tag:raydiant.statuspage.io,2005:/historyRaydiant Status - Incident History2024-03-28T12:25:07ZRaydianttag:raydiant.statuspage.io,2005:Incident/196488512024-01-08T18:44:59Z2024-01-08T18:44:59ZOur authentication partner Auth0 is currently experiencing some degraded performance.<p><small>Jan <var data-var='date'> 8</var>, <var data-var='time'>18:44</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'> 8</var>, <var data-var='time'>18:20</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p>tag:raydiant.statuspage.io,2005:Incident/189715722023-10-30T21:29:13Z2023-10-30T21:29:13ZThere is a disruption in our service. The team is currently investigating.<p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>21:29</var> UTC</small><br><strong>Resolved</strong> - Auth0's outage has been resolved: https://status.auth0.com/incidents/0wnt9fdgq7q3</p><p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>20:22</var> UTC</small><br><strong>Monitoring</strong> - Auth0 service has recovered. Their problem was probably related to AWS having an outage in a region that Auth0 is using for their service.</p><p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>20:18</var> UTC</small><br><strong>Identified</strong> - Our authentication/authorization partner Auth0 is having some technical difficulties.</p><p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>20:08</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:raydiant.statuspage.io,2005:Incident/168875472023-04-18T22:16:47Z2023-04-18T22:16:47ZRaydiant Employee Engagement/Hoopla Services are Unreachable<p><small>Apr <var data-var='date'>18</var>, <var data-var='time'>22:16</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Apr <var data-var='date'>18</var>, <var data-var='time'>18:49</var> UTC</small><br><strong>Update</strong> - Fix has been deployed successfully, the team is now monitoring.</p><p><small>Apr <var data-var='date'>18</var>, <var data-var='time'>17:02</var> UTC</small><br><strong>Update</strong> - The team is pushing a fix now to remediate the last issues.</p><p><small>Apr <var data-var='date'>17</var>, <var data-var='time'>17:50</var> UTC</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Apr <var data-var='date'>17</var>, <var data-var='time'>14:27</var> UTC</small><br><strong>Monitoring</strong> - Some customers are reporting partial outages with some of EX connectors.</p><p><small>Apr <var data-var='date'>17</var>, <var data-var='time'>14:13</var> UTC</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Apr <var data-var='date'>17</var>, <var data-var='time'>14:13</var> UTC</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Apr <var data-var='date'>17</var>, <var data-var='time'>14:11</var> UTC</small><br><strong>Update</strong> - A fix is being deployed. Some services have already recovered.</p><p><small>Apr <var data-var='date'>17</var>, <var data-var='time'>06:08</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:raydiant.statuspage.io,2005:Incident/168094172023-04-10T18:35:13Z2023-04-10T18:35:13ZMultiple Raydiant Services are Unreachable<p><small>Apr <var data-var='date'>10</var>, <var data-var='time'>18:35</var> UTC</small><br><strong>Resolved</strong> - We had a DNS incident causing multiple services to become unavailable. Most should now be reachable while the rest should be soon as the updated DNS settings propagate.</p><p><small>Apr <var data-var='date'>10</var>, <var data-var='time'>14:10</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and services should start to recover soon. We are continuing to monitor.</p><p><small>Apr <var data-var='date'>10</var>, <var data-var='time'>13:14</var> UTC</small><br><strong>Investigating</strong> - We are currently experience an issue with multiple services including Instagram and Calendars. We are actively investigating.</p>tag:raydiant.statuspage.io,2005:Incident/160690222023-02-09T01:35:30Z2023-02-09T01:35:30ZProduction data not visible to users<p><small>Feb <var data-var='date'> 9</var>, <var data-var='time'>01:35</var> UTC</small><br><strong>Resolved</strong> - Service is fully restored, but due to outage with our database, customers that made changes from 4am yesterday may not see those updates reflected. We active are looking at ways to restore any lost changes, but we are unable guarantee all updates will be restored. Please contact support if you have lost updates and would like assistance to restore those changes.</p><p><small>Feb <var data-var='date'> 9</var>, <var data-var='time'>01:19</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Feb <var data-var='date'> 8</var>, <var data-var='time'>23:45</var> UTC</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue.</p><p><small>Feb <var data-var='date'> 8</var>, <var data-var='time'>23:44</var> UTC</small><br><strong>Identified</strong> - User data (i.e. devices, playlist and presentation) is not visible for all users.</p>tag:raydiant.statuspage.io,2005:Incident/151144392022-12-13T22:00:24Z2022-12-13T22:00:24ZIncreased error rates in Menu API<p><small>Dec <var data-var='date'>13</var>, <var data-var='time'>22:00</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Dec <var data-var='date'>13</var>, <var data-var='time'>21:22</var> UTC</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Dec <var data-var='date'>13</var>, <var data-var='time'>21:21</var> UTC</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Dec <var data-var='date'>13</var>, <var data-var='time'>21:11</var> UTC</small><br><strong>Monitoring</strong> - We have corrected the behaviour and are currently monitoring for further errors.</p><p><small>Dec <var data-var='date'>13</var>, <var data-var='time'>21:05</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:raydiant.statuspage.io,2005:Incident/113030532022-09-28T21:28:25Z2022-09-28T21:28:25ZAWS API gateway outage<p><small>Sep <var data-var='date'>28</var>, <var data-var='time'>21:28</var> UTC</small><br><strong>Resolved</strong> - The AWS outage has been resolved and all applications have returned to normal operation. Thank you for your patience.</p><p><small>Sep <var data-var='date'>28</var>, <var data-var='time'>18:19</var> UTC</small><br><strong>Monitoring</strong> - Apps backend might be effected due the AWS API gateway degradations. <br />Apps that we are aware of that might not be working for some users are Hoopla, Toast, Instagram<br /><br />AWS API gateway status can be found here: https://aws-status.info/</p>tag:raydiant.statuspage.io,2005:Incident/112246682022-09-22T20:00:31Z2022-09-22T20:00:31Zlogin_EX_hoopla_production_env.<p><small>Sep <var data-var='date'>22</var>, <var data-var='time'>20:00</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Sep <var data-var='date'>22</var>, <var data-var='time'>19:55</var> UTC</small><br><strong>Investigating</strong> - EX --> login_EX_hoopla_production_env is FAILING</p>tag:raydiant.statuspage.io,2005:Incident/108861542022-08-22T20:09:24Z2022-08-26T19:51:15Zlogin_EX_hoopla_production_env.<p><small>Aug <var data-var='date'>22</var>, <var data-var='time'>20:09</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'>22</var>, <var data-var='time'>20:06</var> UTC</small><br><strong>Investigating</strong> - EX --> login_EX_hoopla_production_env is FAILING</p>tag:raydiant.statuspage.io,2005:Incident/108848412022-08-22T17:14:00Z2022-08-26T19:51:15Zlogin_EX_hoopla_production_env.<p><small>Aug <var data-var='date'>22</var>, <var data-var='time'>17:14</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'>22</var>, <var data-var='time'>16:36</var> UTC</small><br><strong>Investigating</strong> - EX --> login_EX_hoopla_production_env is FAILING</p>tag:raydiant.statuspage.io,2005:Incident/108694292022-08-20T20:30:47Z2022-08-26T19:51:15Zlogin_EX_hoopla_production_env.<p><small>Aug <var data-var='date'>20</var>, <var data-var='time'>20:30</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'>20</var>, <var data-var='time'>20:21</var> UTC</small><br><strong>Investigating</strong> - EX --> login_EX_hoopla_production_env is FAILING</p>tag:raydiant.statuspage.io,2005:Incident/107997872022-08-10T20:34:05Z2022-08-26T19:51:15ZProduction API outage<p><small>Aug <var data-var='date'>10</var>, <var data-var='time'>20:34</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'>10</var>, <var data-var='time'>20:08</var> UTC</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Aug <var data-var='date'>10</var>, <var data-var='time'>20:08</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Aug <var data-var='date'>10</var>, <var data-var='time'>20:00</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:raydiant.statuspage.io,2005:Incident/103745162022-06-22T03:37:15Z2022-08-26T19:51:15ZAPI Check-Auth EX Hoopla Staging.<p><small>Jun <var data-var='date'>22</var>, <var data-var='time'>03:37</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jun <var data-var='date'>22</var>, <var data-var='time'>03:13</var> UTC</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Jun <var data-var='date'>22</var>, <var data-var='time'>00:24</var> UTC</small><br><strong>Investigating</strong> - Check API Check-Auth EX Hoopla Staging is FAILING</p>tag:raydiant.statuspage.io,2005:Incident/103743652022-06-22T03:37:01Z2022-08-26T19:51:16ZAPI Check-Auth EX Hoopla Staging.<p><small>Jun <var data-var='date'>22</var>, <var data-var='time'>03:37</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jun <var data-var='date'>22</var>, <var data-var='time'>03:13</var> UTC</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Jun <var data-var='date'>21</var>, <var data-var='time'>23:54</var> UTC</small><br><strong>Investigating</strong> - Check API Check-Auth EX Hoopla Staging is FAILING</p>tag:raydiant.statuspage.io,2005:Incident/103740292022-06-22T03:14:22Z2022-08-26T19:51:16ZAPI Check-Auth EX Hoopla Staging.<p><small>Jun <var data-var='date'>22</var>, <var data-var='time'>03:14</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jun <var data-var='date'>21</var>, <var data-var='time'>22:34</var> UTC</small><br><strong>Investigating</strong> - Check API Check-Auth EX Hoopla Staging is FAILING</p>tag:raydiant.statuspage.io,2005:Incident/103740402022-06-22T03:14:07Z2022-08-26T19:51:16ZAPI Check-Auth EX Hoopla Integration.<p><small>Jun <var data-var='date'>22</var>, <var data-var='time'>03:14</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jun <var data-var='date'>21</var>, <var data-var='time'>22:36</var> UTC</small><br><strong>Investigating</strong> - Check API Check-Auth EX Hoopla Integration is FAILING</p>tag:raydiant.statuspage.io,2005:Incident/101173422022-05-31T19:19:57Z2022-08-26T19:51:16ZCheckly webhook test check has failed<p><small>May <var data-var='date'>31</var>, <var data-var='time'>19:19</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>May <var data-var='date'>31</var>, <var data-var='time'>18:37</var> UTC</small><br><strong>Investigating</strong> - Something broke :(</p>tag:raydiant.statuspage.io,2005:Incident/99138492022-05-04T18:00:56Z2022-08-26T19:51:16ZCX Dashboard and API are down<p><small>May <var data-var='date'> 4</var>, <var data-var='time'>18:00</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>May <var data-var='date'> 4</var>, <var data-var='time'>18:00</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implement3ed and we are monitring results.</p><p><small>May <var data-var='date'> 4</var>, <var data-var='time'>17:45</var> UTC</small><br><strong>Identified</strong> - The issue has been identified and we are workjng on a fix.</p>tag:raydiant.statuspage.io,2005:Incident/99129512022-05-04T16:07:27Z2022-08-26T19:51:16ZCX Weather API is temporarily down<p><small>May <var data-var='date'> 4</var>, <var data-var='time'>16:07</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>May <var data-var='date'> 4</var>, <var data-var='time'>16:07</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring.</p><p><small>May <var data-var='date'> 4</var>, <var data-var='time'>16:05</var> UTC</small><br><strong>Identified</strong> - The issue has been identified and we are working on resolution.</p><p><small>May <var data-var='date'> 4</var>, <var data-var='time'>15:28</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:raydiant.statuspage.io,2005:Incident/97806222022-04-14T18:26:35Z2022-08-26T19:51:16ZCX Free Signup unavailable<p><small>Apr <var data-var='date'>14</var>, <var data-var='time'>18:26</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Apr <var data-var='date'>14</var>, <var data-var='time'>06:01</var> UTC</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Apr <var data-var='date'>14</var>, <var data-var='time'>06:01</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:raydiant.statuspage.io,2005:Incident/93159022022-02-14T20:12:31Z2022-08-26T19:51:16ZAPI Code Deployment<p><small>Feb <var data-var='date'>14</var>, <var data-var='time'>20:12</var> UTC</small><br><strong>Resolved</strong> - The deployment has completed successfully.</p><p><small>Feb <var data-var='date'>14</var>, <var data-var='time'>20:01</var> UTC</small><br><strong>Monitoring</strong> - We are rolling out an update to the core Raydiant API which includes new flagged features related to Custom Fonts and Themes background changes. There is no impact expected to customers. We are monitoring the rollout.</p>tag:raydiant.statuspage.io,2005:Incident/91862622022-01-29T00:29:30Z2022-08-26T19:51:16ZProduction outage of music from SoundtrackYourBrand<p><small>Jan <var data-var='date'>29</var>, <var data-var='time'>00:29</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'>29</var>, <var data-var='time'>00:24</var> UTC</small><br><strong>Monitoring</strong> - A fix has been deployed to all affected devices and the outage has been resolved. We will continue to monitor.</p><p><small>Jan <var data-var='date'>28</var>, <var data-var='time'>23:36</var> UTC</small><br><strong>Update</strong> - A fix has been implemented and we are beginning deployment. ETA is +1 hour.</p><p><small>Jan <var data-var='date'>28</var>, <var data-var='time'>21:15</var> UTC</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue and will be testing a fix soon.</p><p><small>Jan <var data-var='date'>28</var>, <var data-var='time'>18:00</var> UTC</small><br><strong>Identified</strong> - The root cause of the issue has been identified and we are working to resolve it. Service will be restored as soon as possible.</p><p><small>Jan <var data-var='date'>28</var>, <var data-var='time'>17:59</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue, and have assigned multiple production engineers as well as an incident manager. This outage only affects customers using SoundtrackYourBrand.</p>tag:raydiant.statuspage.io,2005:Incident/90861352022-01-15T03:44:53Z2022-08-26T19:51:16ZNew custom themes breaking presentations<p><small>Jan <var data-var='date'>15</var>, <var data-var='time'>03:44</var> UTC</small><br><strong>Resolved</strong> - The incident has been resolved and all systems are operating normally.</p><p><small>Jan <var data-var='date'>15</var>, <var data-var='time'>03:44</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring.</p><p><small>Jan <var data-var='date'>15</var>, <var data-var='time'>01:06</var> UTC</small><br><strong>Identified</strong> - The issue has been identified and a fix is underway.</p><p><small>Jan <var data-var='date'>15</var>, <var data-var='time'>01:06</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating the issue.</p>tag:raydiant.statuspage.io,2005:Incident/90286892022-01-07T21:58:55Z2022-08-26T19:51:16ZProduction API outage<p><small>Jan <var data-var='date'> 7</var>, <var data-var='time'>21:58</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'> 7</var>, <var data-var='time'>21:42</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:raydiant.statuspage.io,2005:Incident/88985402021-12-20T22:46:27Z2022-08-26T19:51:16ZPLANNED: Upgrading production database<p><small>Dec <var data-var='date'>20</var>, <var data-var='time'>22:46</var> UTC</small><br><strong>Resolved</strong> - Upgrade is complete. No issues encountered.</p><p><small>Dec <var data-var='date'>20</var>, <var data-var='time'>22:44</var> UTC</small><br><strong>Monitoring</strong> - Upgrade has completed and is being monitored.</p><p><small>Dec <var data-var='date'>20</var>, <var data-var='time'>22:00</var> UTC</small><br><strong>Identified</strong> - Upgrade is in progress.</p><p><small>Dec <var data-var='date'>20</var>, <var data-var='time'>21:59</var> UTC</small><br><strong>Investigating</strong> - We are currently upgrading the production database. There will be a small interruption in services.</p>