#dev | Logs for 2021-03-31

« return
[00:16:27] <TheMightyBuzzard> Bytram, sodium isn't even being used for anything right now. you're thinking hydrogen if you want the other web frontend. bookmark the wiki servers page.
[00:17:55] <Bytram> only sodium when I ran my script which queried free disk space on all the servers. chromas pointed out that things seemed tight on sodium.
[00:20:39] <TheMightyBuzzard> hit preview again
[00:20:59] <Bytram> need to start a fresh page; hold on
[00:21:36] <Bytram> OK
[00:21:36] <Bytram> The server encountered an internal error or misconfiguration and was unable to complete your request.
[00:21:36] <Bytram> Please contact the server administrator, admin@soylentnews.org and inform them of the time the error occurred, and anything you might have done that may have caused the error.
[00:21:36] <Bytram> More information about this error may be available in the server error log.
[00:22:12] <Bytram> (that was from starting in a brand new firefox window)
[00:22:23] <Bytram> TheMightyBuzzard: ^^^
[00:22:48] <TheMightyBuzzard> did you by chance change any of the regexes while you were doing stuff today? not the 999, the regex itself.
[00:22:57] <Bytram> nope
[00:23:08] <Bytram> you saw the SQL I used
[00:23:19] <TheMightyBuzzard> cause it's one of the regexes breaking stuff
[00:23:46] <Bytram> really?
[00:24:24] <Bytram> Nope, I did some queries but o/w the only change as as you saw copy/pased into IRC
[00:24:56] <TheMightyBuzzard> hit preview again
[00:25:21] <Bytram> not that easy. gotta start fresh
[00:25:54] <Bytram> SAM AS BEFORE
[00:26:17] <TheMightyBuzzard> again
[00:26:27] <TheMightyBuzzard> you don't have a back button?
[00:27:42] <Bytram> I do always an to start frm a known good condition
[00:27:52] <TheMightyBuzzard> don't. if i need it i'll ask
[00:28:03] <Bytram> that seemd to work\
[00:28:13] <TheMightyBuzzard> broken regexes
[00:28:49] <TheMightyBuzzard> btw, you're not getting load balanced to fluorine, you're getting load balanced to hydrogen
[00:28:57] <Bytram> what filter #
[00:29:39] <TheMightyBuzzard> absent filter #s
[00:29:44] <TheMightyBuzzard> deleted
[00:30:30] <Bytram> oh, #35 and #36
[00:30:48] <Bytram> if memory serves
[00:30:48] <TheMightyBuzzard> i'd rather have spam than be down, so buggy filters get deleted and then replaced correctly if necessary
[00:30:58] <Bytram> nod nod
[00:31:13] <TheMightyBuzzard> bout 7 of them. all of them with \bword\b shape
[00:31:58] <Bytram> k
[00:32:34] <Bytram> so... curious why they suddenly start acting up?
[00:32:49] <Bytram> things seemed fine for a long long time with them there
[00:35:17] * TheMightyBuzzard shurgs
[00:35:43] * Bytram shurgs, two
[00:36:31] <TheMightyBuzzard> that stuff you pasted from the logs wasn't you btw. wrong box.
[00:36:56] <Bytram> which stuff?
[00:37:09] <TheMightyBuzzard> um, the only stuff you pasted from the logs
[00:37:44] <Bytram> comment_filters? that was from fluorine as far as I knew
[00:38:08] <TheMightyBuzzard> no... in this channel, two hours ago
[00:39:56] <Bytram> you mean: tail -f /srv/soylentnews.org/apache/logs/error_log?
[00:40:01] <Bytram> etc?
[00:41:32] <TheMightyBuzzard> yep. you weren't hitting fluorine with your browser, you were hitting hydrogen.
[00:42:43] <Bytram> that was where I thught I was supposed to be, but then again -- all this happened just beforre I was about to take a nap. was FAR from being in my best shape... and still need that nap.
[00:42:53] <TheMightyBuzzard> nod nod
[00:43:26] <TheMightyBuzzard> for future reference, works like this. traffic hits whichever load balancer we're using. it's magnesium unless something's up.
[00:43:54] <TheMightyBuzzard> then it gets delegated to either hydrogen or fluorine for the actual page stuff. load balanced.
[00:44:28] <TheMightyBuzzard> both have their own separate logs and you'll only be in whichever one you're hitting currently.
[00:54:09] <Bytram> TheMightyBuzzard: sry, was off writing this up this incident; could you please take a look, make any needed changes, and then toggle it t be DISPLAY?
[00:54:20] <Bytram> https://soylentnews.org
[00:54:34] <Bytram> short break
[00:57:32] <TheMightyBuzzard> eh, i've got a few ideas of how it could have happened. i just don't care enough to find out which it was since none of them would mean anything for site stability and the offending filters are gone.
[00:59:17] <Bytram> sadly, if you tried to explain it to me right now, I doubt it would even register. I'm wiped
[00:59:36] <Bytram> thanks a TON for getting things sorted!
[00:59:40] <Bytram> TheMightyBuzzard++
[00:59:40] <Bender> karma - themightybuzzard: 31
[00:59:45] <TheMightyBuzzard> it'd take more work to explain than to check and it wasn't worth checking
[01:01:40] <Bytram> if that story is pkay to release, please check it to DISPLAY and hit update
[01:01:43] <Bytram> https://soylentnews.org
[01:02:36] <TheMightyBuzzard> honestly, i don't think it's worth a story. did anyone besides you notice?
[01:03:02] <Bytram> yes, saw a comment in a story at first, and several mentions on IRC
[01:03:31] <Bytram> https://soylentnews.org
[01:04:16] <Bytram> that was about 3.5 hours ago
[01:05:36] <TheMightyBuzzard> ahh, roger. brevity then. something along the lines of "We were retiring some spam filters on submissions and managed to somehow confuse Apache. The offending filters are now deleted entirely and incapable of confusing even chromas."
[01:06:50] <TheMightyBuzzard> doesn't need your browser's error message, that's browser-specific
[01:09:06] <Bytram> oh, it is? I thought that came from the site.
[01:09:18] <Bytram> I like your wrding better
[01:10:45] <Bytram> do what you think is right -- I can't think straight any more. but, we DO owe it to the community to explain what happened. and, Buck Feta! We're better than that other site, right? :)
[01:11:17] <TheMightyBuzzard> yar. i'll poke it into shape after i get some dinner in me.
[01:37:24] <Bytram> oh, geesh! You hadn't eaten yet? Yikes!
[01:37:37] <Bytram> in that case, double thanks!
[01:38:46] * Bytram sees you already updated the story
[01:38:52] * Bytram takes a look
[01:39:07] <Bytram> TheMightyBuzzard: Looks great! Thanks bunches!