Downtime


  • Coder

    Hi; sorry for not posting about this before the event.

    Mechanipus server Kuramori is down for upgrades by my hosting provider (Linode) to their xen hypervisor infrastructure related to a not-yet-disclosed security issue.

    Estimated downtime is less than two hours. The following games are affected:

    arrakis battlemux belleauwood chaoticcosmos clockwork cmuffin cofab cotl crossover darkspires darkwater dcfallen dystopia eldritch fnt goetia gundam hermworld hpb julesletters magicplays masseffect medieval muffinmu nevermore nocturne nolabigeasy oldandnew pony reno rifts spellbound starwars statuvariabilis stromm suicidechilde tgg thedrift uumux veristicheaven whrofh wildcard zero

    Midori and Kuu will follow. Times are, with two-hour windows:

    • 2015-03-05 1:00:00 AM UTC - Kuramori
    • 2015-03-07 3:00:00 AM UTC - Midori
    • 2015-03-08 10:00:00 PM UTC - Kuu

    For the people riding the large elephant in the room, Kuu runs The Reach and various other things.

    There's nothing wrong, your games are fine; was a clean shut down and service will return after they finish their updates.


  • Coder

    So. This is a thing that happened.

    Nearly all games came up fine as I keep the shell cron @reboot lines up to date and I'm pretty anal about information safety and system hygiene with respect to keeping things up-to-date and orderly.

    One guy's stuff didn't. I'm kinda speechless at the level of foil-hat bat-shittery contained in this log. I'd wrap it in a spoiler-tag, but apparently those don't work here yet.

    tl;dr take away lessons:

    • I'm not out to get you.
    • I'm not launching a l33t h4x0r cyberattack on the world, your pets and/or loved ones, or your game.
    • I'm happy to provide basic assistance and handholding within a reasonable timeframe as I like teaching people new things.
    • I do expect people to have a basic sense of self responsibility.
    • Game owners must understand how their games are started, what they do when they don't start, how to access a shell, how to take backups, how to verify backups, etc. etc. etc.
    • Most don't, but can coast by fine if they are running reasonably normal games, have followed past advice, and/or have a unix-savvy friend they can lean on once in a while.
    • The vast majority of people are really nice and a joy to talk to.
    • The other 1% is NOT and can FUCK OFF because I soooo did not need to deal with this shit today.

    And so, here is the log. I tried to help him. Didn't work:


    <a game owner>: alright no problem.. for some reason one of my sites seems to have come up but there's no DB or anything, other siste never came back
    <Chime>: was a clean shutdown, so that's a game config problem?
    <a game owner>: No idea... you setup all my sites
    <Chime>: okay, which game is having what problem?
    <a game owner>: ____.mechanipus has no DB loaded and ______ is not even online
    <Chime>: ______ doesn't have a game/ dir or anything; there's a penn install and an rhost install
    <Chime>: I don't know what you have set up there, but the cron @reboot line in the shell never got updated for that
    <Chime>: should be safe, just needs a restart?
    <Chime>: looking at ____
    <a game owner>: ______ is the rhost
    <a game owner>: I've never started any of these either, you always did it
    <Chime>: Do you have codestaff that can manage these sorts of things?
    <Chime>: I'm trying to step away from game-by-game assistance as much as possible
    <a game owner>: Well I've never done it before and you didn't send any warning this would happen.
    <Chime>: Okay.
    <Chime>: well.
    <Chime>: Let's fix this:
    <Chime>: You log in to the shell and start it
    <Chime>: that way, you'll know how
    <Chime>: and I can answer questions if you have them
    <a game owner>: Don't know how and don't have time to, if I had known it would be going down I could have prepared for it.
    <Chime>: Okay. You're welcome to copy your files off and get hosted elsewhere?
    <a game owner>: Just seems a little rude to take the stuff down without warning and then tell me any problems are my fault when you set it all up
    <a game owner>: Least you could do is put the games back up
    <Chime>: What I set up-- for all the other games-- came back up fine.
    <a game owner>: Not at all mine aren't up and you're the only one who has set them up
    <Chime>: You've gone out of your way to get custom rhost/penn stuff and I can't tell what the startup routine should be without asking you each time
    <a game owner>: You've done it each time, so I'm not sure what you need to ask me
    <Chime>: I should have sent out an alert. Didn't have a chance to, as work has been keeping me busy. Sorry for the inconvenience.
    <a game owner>: Everytime there is some unexpected or unanounced alert mys tuff goes down and there is a problem yet it runs fine the other 100% of the time
    <a game owner>: If you don't want to host me fine, please fix it, so I can let people know to come decompile their crap and then I'll backup my stuff and be done with this nonsense
    <a game owner>: I actually liek to warn people if something is going down
    <Chime>: I like to warn people too.
    <Chime>: I don't always have that opportunity.
    <Chime>: Part of the problem here is I need you to take responsibility for some aspects of game administration
    <Chime>: "You always did it before" is not an okay answer.
    <a game owner>: Like I said, please just put the games up properly so I can warn the players they will be shutting down.
    <Chime>: okay, check ______. I started what seems to be current and put that in cron
    <a game owner>: Like they're literally loaded with blank DBS right now
    <a game owner>: This is the same crap that happened when the servers crashed
    <Chime>: then you've got a buggy mush hardcode and I cannot help you
    <Chime>: your data is probably there, but you're doing something very wrong if this keeps happening
    <a game owner>: Yes you can, you've done it every other time
    <a game owner>: Sadly, you've never told me what you've done the other times, you've just done it so it's not as if I've got notes.
    <Chime>: fine.
    <Chime>: looking closer.
    <a game owner>: Like I said, If you don't want to host me, I'll gladly just close my games down and backup the data but I would like the opportunity for players to grab anything they want or need first etc.. and they cna't do that right now
    <Chime>:

    INFO
    ### Begin INFO 1
    Name: ______ ____
    Uptime: Wed Mar  4 22:12:15 2015
    Connected: 0
    Size: 9990
    Version: RhostMUSH Alpha 3.9.3RL(A).p3
    ### End INFO
    

    <Chime>: that looks up?
    <a game owner>: With no data in the DB loaded when I logon.. no character bits, no staff biuts, limbo is the base room
    <a game owner>: Like I said, you've fixed this every time and not told me what you did so it's not like I have a clue what to tell you to do.
    <Chime>: heh, wow.
    <Chime>: okay.
    <Chime>: as a game owner
    <Chime>: do you understand where your flatfile is
    <Chime>: where your backups are going, etc?
    <Chime>: independent of who set you up or how-- those things are irrevocably your sole responsibility.
    <Chime>: I will try to help
    <Chime>: but come ON
    <a game owner>: Nope, whoever server host is has always set that up; which in this case is you. Thatr's why I said, just put these up so I can let players grab what they want and I will be done.
    <a game owner>: I'm sorry that suddenly you have an unwillingness to help, but that's why I had you host because you were willing to help.
    <Chime>: my willingness to help evaporated months ago because of various life problems. I haven't been taking on new customers and I've been slowly phasing things out.
    <a game owner>: If I had known that you would suddenly take down the sites, then put them back up broken and tell me how terrible I am I could have just went with another host a long time ago. This is why I'm done, if you want, I can just mass email the players, let them know the site is down for good because the person who has been administrating the site since existence told me to go fuck myself when they took it downm
    <Chime>: in particular, doing the full-featured support you're looking for is one of the most singularly unrewarding things in the world.
    <Chime>: I did not take your site down.
    <Chime>: I did not break your site.
    <Chime>: I did not hide how your site is configured, or take your data, or anything.
    <Chime>: I haven't told you to go fuck yourself
    <a game owner>: Everything is always working until these 'unannounced' shutdowns.
    <Chime>: Though I'm very sorely tempted.
    <Chime>: okay
    <Chime>: You are welcome to copy your files off
    <a game owner>: Telling me how terrible I am and how I should know what to do when you've always done it?
    <a game owner>: That's pretty much telling me to go fuck myself
    <a game owner>: I'm not sure what else you call that.
    <a game owner>: Since you've handled the site side of things since I first hosted with you almost 3 years ago.
    <a game owner>: I really don't give a shit if you want to do it for me, but there are a handful of people who still connect to those sites who I am sure would like the chance to get their character data and send out contact info to friends
    <a game owner>: So if you think sabotaging me is cool, awesome. I'd like you to think of those people though and how annoying it is to suddenly just lose contact with others and lose your stuff.
    <a game owner>: Maybe if you had showed me day 1 what the hell you did to fix it after these crashes/shutdowns that always hapen without warning; I would have known what to do. So please, for the love of god, just fix it this one last time and we can be done.
    <Chime>: your next host should be able to salvage anything you need
    <a game owner>: I'm not looking for another host, I'd like to give these people a chance to grab what they like off the site.
    <a game owner>: Like I said, just fix the problem you caused please so I can be done with this crap.
    <Chime>: No.
    <Chime>: I'm done.
    <a game owner>: You have a responsibility as a host unfortunately
    <Chime>: No. I don't.
    <a game owner>: You do.
    <a game owner>: I'm asking you to put back up the sites you took down.
    <a game owner>: It's all well and good to lecture me on what I should know, but unfortunately it's you who has done this since you hosted me.
    <Chime>: We don't seem to be communicating here.
    <a game owner>: Trust me, had I known there was going to be a shutdown and you were going to have a meltdown over me asking you nicely for help; I would have told people beforehand the games are shutting down.
    <Chime>: Your set of assumptions is broken. I cannot help that. Your game is down. My services came up fine, but your specialized config did not. Out of the goodness of my heart, I have assisted with that in the past, and I can see now that this was a terrible mistake.
    <Chime>: You have no desire or willingness to take responsibility for your own game
    <a game owner>: Actually, if you had told me or showed me what to do before I would know.
    <Chime>: This discussion is over.
    <a game owner>: I didn't know because you took responsibility as a host.
    <a game owner>: It's not.
    <a game owner>: I would like you to fix what you have broken please.
    <a game owner>: At this point, this is starting to look like a purposeful attack.
    <a game owner>: Since I have evidence that at every point in the past; you have fufilled duties as a site administrator and fixed the issues after this.
    <a game owner>: Now, after a 'sudden mysterious' downtime that was not warned about or announced in a very irresponsible manner; you are claiming I should know how to fix something you have done up until this point.
    <a game owner>: Cyber-attacks are very serious, especially if you are abusing your position as a server host to do them.
    <a game owner>: What you are doing right now is an attack on me.
    <a game owner>: More importantly though, it is an attack on the people who connect to the sites.
    <a game owner>: I also have multiple emails before of me asking you about payment, which you've never responded about.
    <a game owner>: Now neither site is even booting, even more odd.
    <a game owner>: If you like, feel free to zip up my site files and send them to me so I don't need to logon to your servers since none of my access is working.
    <a game owner>: Thanks so much for all the help over the years, wish I knew it was going to come back to bite me in the ass when you decided you didn't enjoy helping people anymore.
    <a game owner>: God bless.


    I'm not responding to him anymore because I don't see much point. I've since taken down all three of his games (rather than the two that were broken) and closed the accounts and archived the files. It's more than would fit on a CD and kinda a disorganized jumble, because that's how he keeps his data. Judging by his comments and my past recollections it would be fairly easy to fix, but his behavior kinda removed all incentive to do so. If he can send me a postal address, I'll be happy to mail him a DVD-R or something.

    I'm providing what is currently a free service because I DON'T HAVE TIME FOR THIS SHIT. @Loki, I so totally sympathize with why you guys took down ISU. I'm in the same boat, but never had a business partner to help with taxes and billing. Mushhosting is one of the most singularly unrewarding endeavors a person could ever undertake. I think I'd seriously rather go do retail.

    For everyone else I'm still hosting: You're awesome, and thank you. Your stuff will continue to work for as long as you need it. If you have trouble, please just ask nicely and I'll try to help as soon as I can.

    For everyone else who wanted hosting: I'm sorry that I ran out of patience and give-a-damn. I'm quite happy to answer questions between hosting-providers though, and provide guidance for others on how to do these things. But really, my best recommendation is: don't. It's not worth it.


  • Coder

    Chime, I sympathize greatly with you in this conversation, and with your client. It's not a very good situation for either of you, and I'm sorry it happened. I don't have a lot of time, either, these days; however, if you want a backup on support maybe I can give you a hand now and then as needed. I'm happy to help your customers out under the same conditions which you are: when I have time.


  • Pitcrew

    @Chime, you've always been amazing, and frankly the only thing I was sad about was that we never really had a way to pay you for the amazing job you do. If you ever wanna set up like, a patreon or something? I'll subscribe to that shit in a second. The fact that you continue to host everyone astounds me, especially when it has you having to show patience like that to assholes who think they're somehow entitled to anything from a free service.

    You're a fucking saint, and one of the best people in this hobby. I will mourn the day you finally decide to set your whole house and everything in it on fire and retreat from the internet in disgust with us all.


  • Pitcrew

    @RDC said:

    You're a fucking saint, and one of the best people in this hobby. I will mourn the day you finally decide to set your whole house and everything in it on fire and retreat from the internet in disgust with us all.

    Hey! That's MY house! With my stuff in it. And ME in it!


  • Pitcrew

    Yes, well. Presumably you would leave the house. :P


  • Pitcrew

    Leave the house? Never!


  • Pitcrew

    THEN BURN, BWAHAHAHAHAHAH.


  • Coder

    Haha!

    No, I won't be burning the house down, with lemons or otherwise.

    I will be setting up a Patreon in the near future as it sounds like a much much better way of doing things.

    Again, if I ever do need to shut it all down, I'll give people plenty of warning and help moving off. I just... kinda couldn't take that guy anymore :(

    I'll definitely ask @loki for help when needed. Much appreciated!


  • Admin

    @Chime Wait, are you saying this guy is using a free service?


  • Pitcrew

    Chime's service is free, yes.


  • Admin

    While I was reading that log I kept thinking that guy was way ruder than he should be while asking for customer service.

    But if he's not even a customer then all he is is an entitled douchebag.



  • Hehe.. I'm horrible. First thing I did when I saw 'Chime's Service is free' was giggle like a teenage school boy.



  • Poor Chime. :( I can understand being frustrated at the loss, but if you run a game, isn't it just a good idea to keep a flatfile or back up? Maybe I'm dumb...


  • Coder

    Extra classy: he started replying to unrelated G+ posts I'd made.


  • Admin

    @Chime said:

    Extra classy: he started replying to unrelated G+ posts I'd made.

    And that guy is a MU* owner. :) It'd be so cool to play on one of his storied games.



  • @Chime Wow, he sounds like a real winner. :/ I'm sorry.


  • Coder

    @Chime Is there any reason you're not naming this particular MU owner? Or at least the MU itself.


  • Admin

    @Glitch said:

    @Chime Is there any reason you're not naming this particular MU owner? Or at least the MU itself.

    It could save someone here a bit of grief. I can't imagine many of us would want to play where someone of that caliber is staffing.



  • It was me. I get like that some times, especially when I haven't had coffee.

    Yeah, no.

    But I can offer a pint to Chime.