Pre Migration Site Outage

Discussion in 'Announcements, How do I, Feedback' started by Sully, Jul 14, 2017.

  1. Guest - Welcome to the new Motoring Alliance Site! Steve, MCS02 and I were up until the early hours this morning working to get it done... as you can see it took us a bit longer than expected to get Phase 1 up and running. We are not 100% there.. its a journey. For site updates, "How Do I" and feedback please use the Announcements, How Do I, Feedback Forum (Links below). Like building a car.... its never done... and thanks for your patience

    Site Announcements "How Do I" Feedback
  1. Sully

    Sully Administrator
    Staff Member Articles Moderator

    Jul 21, 2015
    449
    49
    28
    All -

    Over the past few weeks, we have been seeing some very odd behavior on the site. Specifically you as members would see random things stop working like profiles, garage updates, etc stop working. On the backend files were randomly going missing. I thought it was resolved last week. :mad2:

    Last night we had a complete site outage starting around 3:30 - 4:00AM. It appears a large number of files disappeared which required me to do a restore from a backup. I'm in the process of completing the restore. Once that happens, I will see if there is a way to restore the post that were created after the backup.

    I currently do not know root cause. As I move through resolving this I will keep you update.

    Please let me know if you have any questions or find functionality that isn’t working.
     
  2. Sully

    Sully Administrator
    Staff Member Articles Moderator

    Jul 21, 2015
    449
    49
    28
    Quick Update -

    The hosting provider said the restore is still in process due to the overall size. I'm working with them to try and restore the missing post from the last 48hrs... still working on it.

    Just wanted to keep everyone in the loop. Let me know if you see any issues etc.
     
  3. Crashton

    Crashton Club Coordinator

    Jun 4, 2009
    17,860
    8,376
    113
    Hooterville Ohio USA
    Thanks for your hard work Sully. :thumbsup:
     
  4. MCS02

    MCS02 Moderator
    Staff Member Articles Moderator Premium Supporter

    Jul 31, 2009
    7,347
    4,001
    113
    Yea thanks! I think I need a restore. I wish I had a backup from my 20's :D
     
  5. vetsvette

    vetsvette Well-Known Member
    Premium Supporter

    Nov 9, 2013
    1,844
    1,261
    113
    South Central Virginia
    I'd settle for my 30's. I don't think I could survive my 20's again.
     
  6. Metalman

    Metalman Well-Known Member
    Lifetime Supporter

    Sep 29, 2009
    12,535
    7,389
    113
    Owner of a small custom metal fabrication company,
    Columbus, Ohio
    Thank's Sully....

    It's Verking again...:thumbsup:
     
  7. Metalman

    Metalman Well-Known Member
    Lifetime Supporter

    Sep 29, 2009
    12,535
    7,389
    113
    Owner of a small custom metal fabrication company,
    Columbus, Ohio
    Weird...

    When I posted the thank you...

    Got an error message
     
  8. Metalman

    Metalman Well-Known Member
    Lifetime Supporter

    Sep 29, 2009
    12,535
    7,389
    113
    Owner of a small custom metal fabrication company,
    Columbus, Ohio
    Still acting weird...
     
  9. Sully

    Sully Administrator
    Staff Member Articles Moderator

    Jul 21, 2015
    449
    49
    28
    Thanks, really hate that it went down. I'm still working on testing different things on the restore.

    Metalman - what type of error did you get?
     
  10. Sully

    Sully Administrator
    Staff Member Articles Moderator

    Jul 21, 2015
    449
    49
    28
    Ok - looks like I see the error.. error 500. I will have to look into that one. I'm still working on the restore, but that wasn't happening yesterday. I wonder if this is because I posted this in the middle of the restore.

    I'll start a "Test" thread and also check others to see if the "500" errors are happening.
     
  11. Sully

    Sully Administrator
    Staff Member Articles Moderator

    Jul 21, 2015
    449
    49
    28
    Does this response work, was getting error 500 on the other responses I did on this thread.
     
  12. Sully

    Sully Administrator
    Staff Member Articles Moderator

    Jul 21, 2015
    449
    49
    28
    OK - this is odd. When you post you get an error 500.. then later the post show up. Working with URLJet on it.
     
  13. Metalman

    Metalman Well-Known Member
    Lifetime Supporter

    Sep 29, 2009
    12,535
    7,389
    113
    Owner of a small custom metal fabrication company,
    Columbus, Ohio
    Same thing you discovered...
     
  14. Sully

    Sully Administrator
    Staff Member Articles Moderator

    Jul 21, 2015
    449
    49
    28
    All - still having backend DB issues. Working on tracking them.. sorry for the up and down.
     
  15. Sully

    Sully Administrator
    Staff Member Articles Moderator

    Jul 21, 2015
    449
    49
    28
    Quick update - yesterday I did a partial restoration hoping I would be able to recover post. At this point, it looks like we will need to do a full restore. (this post will get lost in the process).. but for those who are online, wanted you to know its going to start happening soon.. Again I will try to recover the post and once back up.
     
  16. wjosh45

    wjosh45 Member

    Nov 20, 2016
    116
    0
    16
    Still working on resolving.. I expect a few more issues
     
  17. Sully

    Sully Administrator
    Staff Member Articles Moderator

    Jul 21, 2015
    449
    49
    28
    OK - so gallery is mostly back up... by that I mean the main page, calendar etc. When you click on the profile, doesn't show anything. Still working on that and root cause. :fingerscrossed:
     
  18. Metalman

    Metalman Well-Known Member
    Lifetime Supporter

    Sep 29, 2009
    12,535
    7,389
    113
    Owner of a small custom metal fabrication company,
    Columbus, Ohio
    Sully...

    When you post to a thread.... You get this error message...

    Screen Shot 2017-07-16 at 6.48.41 AM.png

    But the posting does go through anyway...
     
  19. Sully

    Sully Administrator
    Staff Member Articles Moderator

    Jul 21, 2015
    449
    49
    28
    Metalman -

    Thank you, I appreciate the update. I had all of it working last night around 2am except the "Profiles." when launched from the gallery page. Somehow in 8 hours we are now seeing this come back. I'm working with URLjet and Nathan to continue to figure out root cause. Ultimately something is deleting the files, I believe its either a corrupt script or a SQL injection (Its my understanding from Nathan, that is what happened a few years ago that created this). Finding that is like finding a needle in a haystack.. but I'm working on this.

    Again - sorry this is happening, but I will get it figured out. Let me know if you are seeing other issues. I'm also walking through different functionality and logs to find issues.
     
  20. Sully

    Sully Administrator
    Staff Member Articles Moderator

    Jul 21, 2015
    449
    49
    28
    All -

    Quick update and follow up to Metalman's post. When you post you will get an error 500. If you then go back to motoringalliance.com you will see it actually posted. I'm working through that error, but at least it does appear the post does happen. I'm working through this and hope to have it resolved today if possible.
     

Share This Page