#dev | Logs for 2021-03-31

« return
[01:39:07] <Bytram> TheMightyBuzzard: Looks great! Thanks bunches!
[01:38:52] * Bytram takes a look
[01:38:46] * Bytram sees you already updated the story
[01:37:37] <Bytram> in that case, double thanks!
[01:37:24] <Bytram> oh, geesh! You hadn't eaten yet? Yikes!
[01:11:17] <TheMightyBuzzard> yar. i'll poke it into shape after i get some dinner in me.
[01:10:45] <Bytram> do what you think is right -- I can't think straight any more. but, we DO owe it to the community to explain what happened. and, Buck Feta! We're better than that other site, right? :)
[01:09:18] <Bytram> I like your wrding better
[01:09:06] <Bytram> oh, it is? I thought that came from the site.
[01:06:50] <TheMightyBuzzard> doesn't need your browser's error message, that's browser-specific
[01:05:36] <TheMightyBuzzard> ahh, roger. brevity then. something along the lines of "We were retiring some spam filters on submissions and managed to somehow confuse Apache. The offending filters are now deleted entirely and incapable of confusing even chromas."
[01:04:16] <Bytram> that was about 3.5 hours ago
[01:03:31] <Bytram> https://soylentnews.org
[01:03:02] <Bytram> yes, saw a comment in a story at first, and several mentions on IRC
[01:02:36] <TheMightyBuzzard> honestly, i don't think it's worth a story. did anyone besides you notice?
[01:01:43] <Bytram> https://soylentnews.org
[01:01:40] <Bytram> if that story is pkay to release, please check it to DISPLAY and hit update
[00:59:45] <TheMightyBuzzard> it'd take more work to explain than to check and it wasn't worth checking
[00:59:40] <Bender> karma - themightybuzzard: 31
[00:59:40] <Bytram> TheMightyBuzzard++
[00:59:36] <Bytram> thanks a TON for getting things sorted!
[00:59:17] <Bytram> sadly, if you tried to explain it to me right now, I doubt it would even register. I'm wiped
[00:57:32] <TheMightyBuzzard> eh, i've got a few ideas of how it could have happened. i just don't care enough to find out which it was since none of them would mean anything for site stability and the offending filters are gone.
[00:54:34] <Bytram> short break
[00:54:20] <Bytram> https://soylentnews.org
[00:54:09] <Bytram> TheMightyBuzzard: sry, was off writing this up this incident; could you please take a look, make any needed changes, and then toggle it t be DISPLAY?
[00:44:28] <TheMightyBuzzard> both have their own separate logs and you'll only be in whichever one you're hitting currently.
[00:43:54] <TheMightyBuzzard> then it gets delegated to either hydrogen or fluorine for the actual page stuff. load balanced.
[00:43:26] <TheMightyBuzzard> for future reference, works like this. traffic hits whichever load balancer we're using. it's magnesium unless something's up.
[00:42:53] <TheMightyBuzzard> nod nod
[00:42:43] <Bytram> that was where I thught I was supposed to be, but then again -- all this happened just beforre I was about to take a nap. was FAR from being in my best shape... and still need that nap.
[00:41:32] <TheMightyBuzzard> yep. you weren't hitting fluorine with your browser, you were hitting hydrogen.
[00:40:01] <Bytram> etc?
[00:39:56] <Bytram> you mean: tail -f /srv/soylentnews.org/apache/logs/error_log?
[00:38:08] <TheMightyBuzzard> no... in this channel, two hours ago
[00:37:44] <Bytram> comment_filters? that was from fluorine as far as I knew
[00:37:09] <TheMightyBuzzard> um, the only stuff you pasted from the logs
[00:36:56] <Bytram> which stuff?
[00:36:31] <TheMightyBuzzard> that stuff you pasted from the logs wasn't you btw. wrong box.
[00:35:43] * Bytram shurgs, two
[00:35:17] * TheMightyBuzzard shurgs
[00:32:49] <Bytram> things seemed fine for a long long time with them there
[00:32:34] <Bytram> so... curious why they suddenly start acting up?
[00:31:58] <Bytram> k
[00:31:13] <TheMightyBuzzard> bout 7 of them. all of them with \bword\b shape
[00:30:58] <Bytram> nod nod
[00:30:48] <TheMightyBuzzard> i'd rather have spam than be down, so buggy filters get deleted and then replaced correctly if necessary
[00:30:48] <Bytram> if memory serves
[00:30:30] <Bytram> oh, #35 and #36
[00:29:44] <TheMightyBuzzard> deleted
[00:29:39] <TheMightyBuzzard> absent filter #s
[00:28:57] <Bytram> what filter #
[00:28:49] <TheMightyBuzzard> btw, you're not getting load balanced to fluorine, you're getting load balanced to hydrogen
[00:28:13] <TheMightyBuzzard> broken regexes
[00:28:03] <Bytram> that seemd to work\
[00:27:52] <TheMightyBuzzard> don't. if i need it i'll ask
[00:27:42] <Bytram> I do always an to start frm a known good condition
[00:26:27] <TheMightyBuzzard> you don't have a back button?
[00:26:17] <TheMightyBuzzard> again
[00:25:54] <Bytram> SAM AS BEFORE
[00:25:21] <Bytram> not that easy. gotta start fresh
[00:24:56] <TheMightyBuzzard> hit preview again
[00:24:24] <Bytram> Nope, I did some queries but o/w the only change as as you saw copy/pased into IRC
[00:23:46] <Bytram> really?
[00:23:19] <TheMightyBuzzard> cause it's one of the regexes breaking stuff
[00:23:08] <Bytram> you saw the SQL I used
[00:22:57] <Bytram> nope
[00:22:48] <TheMightyBuzzard> did you by chance change any of the regexes while you were doing stuff today? not the 999, the regex itself.
[00:22:23] <Bytram> TheMightyBuzzard: ^^^
[00:22:12] <Bytram> (that was from starting in a brand new firefox window)
[00:21:36] <Bytram> More information about this error may be available in the server error log.
[00:21:36] <Bytram> Please contact the server administrator, admin@soylentnews.org and inform them of the time the error occurred, and anything you might have done that may have caused the error.
[00:21:36] <Bytram> The server encountered an internal error or misconfiguration and was unable to complete your request.
[00:21:36] <Bytram> OK
[00:20:59] <Bytram> need to start a fresh page; hold on
[00:20:39] <TheMightyBuzzard> hit preview again
[00:17:55] <Bytram> only sodium when I ran my script which queried free disk space on all the servers. chromas pointed out that things seemed tight on sodium.
[00:16:27] <TheMightyBuzzard> Bytram, sodium isn't even being used for anything right now. you're thinking hydrogen if you want the other web frontend. bookmark the wiki servers page.