#dev | Logs for 2015-06-01
« return
[01:00:55] -!- Bender has quit [Remote host closed the connection]
[01:01:22] -!- Bender [Bender!Bender@Soylent/BotArmy] has joined #dev
[01:16:07] -!- Bender has quit [Remote host closed the connection]
[01:17:14] -!- Bender [Bender!Bender@Soylent/BotArmy] has joined #dev
[01:19:17] -!- Bender has quit [Remote host closed the connection]
[01:19:36] -!- Bender [Bender!Bender@Soylent/BotArmy] has joined #dev
[01:21:33] -!- Bender has quit [Remote host closed the connection]
[01:21:49] -!- Bender [Bender!Bender@Soylent/BotArmy] has joined #dev
[01:30:58] -!- Bender has quit [Remote host closed the connection]
[01:31:15] -!- Bender [Bender!Bender@Soylent/BotArmy] has joined #dev
[01:38:01] -!- Bender has quit [Remote host closed the connection]
[01:38:18] -!- Bender [Bender!Bender@Soylent/BotArmy] has joined #dev
[01:52:48] -!- Bender has quit [Remote host closed the connection]
[01:53:10] -!- Bender [Bender!Bender@Soylent/BotArmy] has joined #dev
[01:55:43] -!- Bender has quit [Remote host closed the connection]
[01:56:06] -!- Bender [Bender!Bender@Soylent/BotArmy] has joined #dev
[03:10:53] <Bytram> hi everyone!
[03:11:04] -!- cmn32480 [cmn32480!~cmn32480@Soylent/Staff/Editor/cmn32480] has joined #dev
[03:11:08] <cmn32480> Bytram
[03:11:22] <cmn32480> I've never been here before....
[03:11:27] <Bytram> prod just came back up after being loaded with rehash for the first time
[03:11:39] <Bytram> it's pretty spacious; make yourself at home!
[03:12:00] <cmn32480> kinda quite... almost relaxing
[03:12:11] <Bytram> I had a couple messages waiting for me on prod; two moderation msgs; was able to read them and delete the messages
[03:12:14] <cmn32480> quiet... stupid fingers
[03:12:18] <Bytram> quite!
[03:12:21] <Bytram> =)
[03:12:24] -!- juggs [juggs!~juggs@Soylent/Staff/IRC/juggs] has joined #dev
[03:12:38] <cmn32480> have been able to preview a story and make some minor adjustments.
[03:12:43] * Bytram edits the site down story
[03:14:57] <cmn32480> preview and the story admin fuinctionality seem to be the same
[03:15:29] <cmn32480> <hr> and <small> work properly in preview
[03:16:03] <takyon> how many name="email" <input> text boxes are there
[03:16:10] <Bytram> preview and update of NCommander's story worked okay: https://soylentnews.org
[03:16:10] <aqu4> ^ "3Site Downtime Tonight: Upgrading to Rehash *STILL IN PROGRESS* - SoylentNews"
[03:16:27] * Bytram missed the title... brb
[03:16:39] <NCommander> Its working
[03:16:40] <NCommander> But slowly
[03:16:43] <NCommander> its not using memcache
[03:17:07] <NCommander> There we go
[03:17:08] <Bytram> new title: "Site Downtime Tonight: Upgrade to Rehash Completed"
[03:17:11] <NCommander> THAT made a huge difference
[03:17:18] <NCommander> Bytram, I'm not done updating crap
[03:17:42] <takyon> collapsed comments just went from seconds to load to instant
[03:17:49] <takyon> ^ not scientific
[03:17:53] * Bytram will reupdate title
[03:17:57] <Bytram> and story
[03:18:15] <cmn32480> editing process looks good. No hangups.
[03:18:26] <cmn32480> checking submission next
[03:19:48] <NCommander> takyon, that was due to memcache
[03:19:56] <takyon> I believe you
[03:20:04] <NCommander> takyon, they're going to still be a bit wonky, all the caches in both the DB and in memcache got zeroed out
[03:20:20] <NCommander> MySQL cluster's caches take awhile to populate, it will get considerably better as people pound on the site
[03:22:39] <cmn32480> side thought... are we going to change the story heading from slash to rehash?
[03:23:45] * Bytram thought that was already available?
[03:23:57] <cmn32480> submission went into the queue with no problems.
[03:24:04] <Bytram> nice!
[03:24:09] * Bytram will try a journal entry
[03:24:31] <Bytram> while I'm on about that, how about a new poll? any ideas?
[03:24:32] <cmn32480> i didn't see it Bytram... but that doesn't mean it ain't there.
[03:24:56] <Bytram> nodnod
[03:25:30] <cmn32480> poll - how many bugs will be found in week 1 on rehash?
[03:25:37] <NCommander> Looks like the site is mostly functional
[03:25:39] <NCommander> Just very slow
[03:25:44] <Bytram> agreed
[03:25:53] <cmn32480> 5, 25, 50, more than the grains of sand on a beach
[03:25:59] * Bytram doesn't really notice the slowness
[03:26:52] <cmn32480> ncommander - somebody very smart told us he was certain that would improve as the caches filled. I think we ought to trust him.
[03:27:20] <juggs> NCommander, would it help if we pounded clicks on stories and comments to populate the cache?
[03:27:34] <cmn32480> rehash is available as a story topic
[03:27:43] <juggs> I have a mouse, I can click :D
[03:30:08] <juggs> Homepage is taking ~18 seconds "Waiting for soylentnews.org...." for me on a ctrl-F5
[03:30:14] <NCommander> juggs, I'm working on it
[03:30:26] <NCommander> one of the DB nodes gave up the ghost due to memory issues
[03:30:41] <takyon> http://soylentnews.org
[03:30:42] <aqu4> ^ "3SoylentNews Submissions"
[03:30:45] <takyon> ^ notes aren't saved
[03:31:02] <NCommander> -_-;
[03:31:12] <paulej72> juggs: I think hydrogen still has it gremlins and is not running well enough for the site. We are going to spin up fluorine with rehash like it was before
[03:31:35] <NCommander> yeah, I'm getting on that right now
[03:31:47] <paulej72> takyon: probably db was not updated for the new fields
[03:32:00] <NCommander> Ugh
[03:32:32] * NCommander is installing updates on fluorine right now
[03:32:39] * Bytram just clicked save on a new journal story
[03:32:52] <NCommander> paulej72, I thought it was just gluster that had issues on hydrogen
[03:33:07] * Bytram just hit Ctrl+F5 on the main page
[03:33:21] <cmn32480> we can work around the notes issue for now.
[03:33:42] <cmn32480> merge works
[03:33:47] * NCommander is getting fluorine up as quickly as possible
[03:33:47] <paulej72> Nope without gluster it still had issues. That is another reason I wanted to test on hydrogen before deploying prod
[03:33:57] <NCommander> paulej72, I feel like an epic idiot
[03:34:08] <NCommander> feel free to tell me "I told you so" for the next six months
[03:34:44] <Bytram> NCommander: I am no longer seeing a list of 'Most Recent Journal Entries' on the site's main page.
[03:35:14] * Bytram goes to check his settings
[03:35:17] <paulej72> slash probably has not updated that yet
[03:35:27] <cmn32480> 503 Error
[03:35:38] <NCommander> cmn32480, hydrogen is straining under load
[03:35:40] <takyon> https://www.youtube.com
[03:35:41] <aqu4> ^ "3sad violin montageparodi.mp420 (HD) (i cri evrytim theme song) - YouTube"
[03:35:44] <Bytram> paulej72: that's not kept in the DB?
[03:35:52] <cmn32480> XID: 2097970172
[03:35:53] <paulej72> yes and no
[03:35:55] <cmn32480> understood
[03:36:11] * NCommander is working on getting fluorine up
[03:36:15] <NCommander> I need to get its database frontend up
[03:36:31] <cmn32480> Bytram - Journals Rehashbox shows on main page
[03:36:46] * Bytram hits F5
[03:37:04] * Bytram doesn't see it; tries Ctrl+F5
[03:38:02] <cmn32480> I ahve all my boxes turned on adn they are all there
[03:38:42] <Bytram> HEY! It looks like some of my settings got changed? Customize Slashboxes on https://soylentnews.org are all reset except for 'Use Slashboxes'
[03:38:42] <aqu4> ^ "3Log In"
[03:39:15] <NCommander> Bytram, we didn't change anything there
[03:39:34] <Bytram> Really?!!? Hrrrm.
[03:40:09] <juggs> paulej72, just what the hell is up with hydrogen? Isn't that the one that has been a thorn in our side for... well, always? Can we take it out back and put a bullet in it once and for all and bury it, then dance on its grave? Spin up some magnesium to really get the fire going here :D
[03:40:29] <paulej72> Bytram: prefs for slashboxes are fuxked up
[03:40:36] <Bytram> that's what I'm seeing
[03:41:02] <paulej72> juggs: yes we will do a nuke and repave on hydrogen
[03:41:09] <Bytram> doesn't help that I *think* I just made some changes in my prefs and then clicked on the 'reset back to defaults' button instead of save. :/
[03:41:25] * Bytram just gotr a 503 on the main page
[03:41:28] <paulej72> Bytram: it seems that all of the slashboxes are gone
[03:41:38] <paulej72> from the prefs
[03:41:48] <Bytram> I tried again and now got the site to show
[03:42:40] <Bytram> paulej72: You mean under 'Customize Slashboxes' on this page: https://soylentnews.org
[03:42:40] <aqu4> ^ "3 - SoylentNews User "
[03:43:02] <cmn32480> dollar figures on the subscription page within my user preferneces are too low
[03:43:13] <cmn32480> $4, $12, and $20
[03:43:21] * Bytram enables 'Older Stuff', 'Most Reject Jounal Entries', 'Poll', and 'Quick Links'
[03:43:33] * Bytram clicks save button
[03:44:05] * Bytram closes a half-dozen other SN tabs leaving only the main page up in one tab
[03:44:36] * Bytram does not see any of the items he selected on the main page
[03:45:59] * Bytram reloads SN prefs and notices his selections are not checked any more and is certain that he checked those four and clicked the 'Save' button.
[03:46:20] * Bytram selects 'Older Stuff' and clicks Save
[03:46:40] <Bytram> okay, that *looks* like it took
[03:47:02] * Bytram loads main page again
[03:47:28] <Bytram> okay I see older stuff displayedon the main page!
[03:47:51] * Bytram opens prefs again: https://soylentnews.org
[03:47:57] <Bytram> homepage tab
[03:48:31] * Bytram selects 'Most Recent Journal Entries' and clicks save
[03:49:11] * Bytram now sees the MRJE slashbox
[03:49:44] <cmn32480> Bytram... I can't get into the meat of the preferences.
[03:50:00] * Bytram selected 'Poll' and clicked save button
[03:50:10] * Bytram reloads main page
[03:50:17] <Bytram> cmn32480: which tab?
[03:50:20] <cmn32480> preferences.... and every button under it takes me to the journal
[03:50:58] <Bytram> got 503 while tring to load prefs page
[03:51:02] <cmn32480> password takes em to the right screen
[03:51:20] * Bytram now sees the poll display3ed.
[03:51:28] <cmn32480> messages takes me to the right place
[03:51:44] <cmn32480> and subscriptions takes me to the right place
[03:53:02] * Bytram selects 'Quick Links' on prefs page and clicks Save
[03:53:19] <cmn32480> tabs across the top seem to work correctlyas far as display
[03:53:24] * Bytram reloads main page
[03:53:57] <Bytram> HEY! Now all my other pref choices disappeared again!
[03:55:14] * Bytram again selects 'Older Stuff', 'MOst Recent Journal Entries', and 'Poll'; clicks on Save
[03:56:49] * juggs backs away cautiously..... that looks like a claymore to me :D
[03:58:50] <cmn32480> gentlemen - I have to bail
[03:59:04] <cmn32480> I gotta be at work in <6 hours
[03:59:26] <Bytram> cmn32480: I'm not gonna be much longer etiher... thanks for ALL the help!!!!!!
[03:59:50] <cmn32480> NCommander Paulej72 - you guys are serious kick ass
[04:01:13] -!- cmn32480 has quit [Quit: See You Later]
[04:01:40] <Bytram> NCommander: paulej72: selecting 'Quick Links' in preferences seems to be erasing other 'Customize Slashboxes' choices after clicking on save
[04:02:09] <paulej72> there is some voodoo going on with slashboxes. I did not see this on dev, so it has something to do with the defult box selection code that has gone fubared
[04:02:18] <Bytram> and that's all I got folks. Hope the rollot contines smoothly!
[04:03:19] <NCommander> fluorine ready to go into service
[04:03:35] * NCommander can't get into linode to change the nodebalancer
[04:03:36] <NCommander> paulej72, ^
[04:03:53] <paulej72> OK I'll do it
[04:06:26] <paulej72> NCommander: take a look at the spew coming from the apache logs
[04:07:00] <NCommander> paulej72, on hydrogen?
[04:07:13] <paulej72> flourine right now
[04:08:05] <NCommander> That looks related to slashboxes
[04:09:25] <NCommander> paulej72, I didn't see this on dev before
[04:09:26] <paulej72> some shit broken. it looks like _genericGetCache is failing
[04:09:49] <NCommander> but at least site performance went from abymisal to "eh"
[04:39:06] <takyon> good call on limiting the admin sessions list
[04:56:37] <paulej72> NCommander: why is nginx running on fluorine?
[04:59:21] <NCommander> paulej72, I have no idea
[04:59:38] <NCommander> paulej72, so part of the issue was that the skins table entry for the main site was buggered
[05:01:05] <NCommander> getSection isn't returning an id number for whatever reason
[05:01:08] <NCommander> causing shit to break
[05:01:43] <NCommander> But I still can't figure out why the homepage isn't showing blocks
[05:04:10] <paulej72> i can't seem to gt ipn to run on flourine
[05:04:38] <paulej72> well it is running, but the nodebalancer does not see it
[05:07:50] <NCommander> paulej72, well, I think this is related to skin _none showing up on admin.pl; I'm seeing that on dev
[05:07:52] <NCommander> as well
[05:08:06] <NCommander> I think because we have other nexii defined, its preventing total breakage on dev, but I think that's the root cause
[05:12:09] <paulej72> NCommander: define meta nexus
[05:12:31] <paulej72> images should be in there aleady
[05:12:40] <NCommander> paulej72, well, I want to understand why this is failing
[05:12:50] <NCommander> paulej72, I have no idea when shit started saying _none on dev
[05:12:55] <NCommander> It was mainp on production pre-upgrade
[05:13:22] <NCommander> woohoo
[05:13:26] <NCommander> I just reproduced on dev \o/
[05:13:42] <NCommander> [Mon Jun 01 05:12:58 2015] [notice] Apache/2.2.29 (Unix) DAV/2 mod_apreq2-20090110/2.8.0 mod_perl/2.0.8 Perl/v5.20.1 configured -- resuming normal operations
[05:13:42] <NCommander> getSection skin for 'mainpage' is: $VAR1 = undef;
[05:13:42] <NCommander> _genericGetCache called with table=blocks, table_prime=bid, and undef id: /srv/soylentnews.org/rehash/site/soylent-mainpage/htdocs/article.pl ModPerl::ROOT::ModPerl::Registry::srv_soylentnews_2eorg_rehash_site_soylent_2dmainpage_htdocs_article_2epl line 174 Slash::DB::MySQL::getBlock /srv/soylentnews.org/rehash/site/soylent-mainpage/htdocs/article.pl ModPerl::ROOT::ModPerl::Registry::srv_soylentnews_2eorg_rehash_site_soylent_2dmainpage
[05:13:43] <NCommander> _htdocs_article_2epl line 247 ModPerl::ROOT::ModPerl::Registry::srv_soylentnews_2eorg_rehash_site_soylent_2dmainpage_htdocs_article_2epl::main /srv/soylentnews.org/perl/lib/perl5/site_perl/5.20.1/x86_64-linux-thread-multi/ModPerl/RegistryCooker.pm ModPerl::RegistryCooker line 206 ModPerl::ROOT::ModPerl::Registry::srv_soylentnews_2eorg_rehash_site_soylent_2dmainpage_htdocs_article_2epl::handler /srv/soylentnews.org/perl/lib/perl5/site_p
[05:13:48] <NCommander> erl/5.20.1/x86_64-linux-thread-multi/ModPerl/RegistryCooker.pm ModPerl::RegistryCooker line 206 (eval) /srv/soylentnews.org/perl/lib/perl5/site_perl/5.20.1/x86_64-linux-thread-multi/ModPerl/RegistryCooker.pm ModPerl::RegistryCooker line 172 ModPerl::RegistryCooker::run
[05:15:33] <NCommander> w.t.f
[05:15:35] * NCommander blinks
[05:15:41] <NCommander> It uses the skin name from the skincolors table
[05:15:44] <NCommander> NOT the skins table
[05:15:46] <paulej72> sure, It probably has something to do with reworing getSkin for nexus support
[05:16:47] * NCommander is trying to dig through this code
[05:17:05] <paulej72> getSection never worked with it
[05:18:57] <NCommander> I ... have no idea what I did
[05:19:05] <NCommander> But getSection is now returing proper info on dev
[05:53:09] * juggs buys NCommander a beer
[06:18:55] -!- stormwyrm [stormwyrm!~stormwyrm@173.199.jr.qj] has joined #dev
[06:19:37] -!- stormwyrm has quit [Client Quit]
[06:55:11] -!- janrinok [janrinok!~janrinok@Soylent/Staff/Editor/janrinok] has joined #dev
[06:56:36] <janrinok> I know you guys are working on it - but the stories list has just emptied. On refresh it contains only 2 stories - the upgrade story and the SN Fund story. All the others are missing
[06:57:59] <janrinok> nexus problems?
[07:31:33] janrinok is now known as janrinok|afk
[07:57:17] <FatPhil> looks like there's still dev settings on the live site
[07:58:07] <FatPhil> the css for the logo points to: transparent url(//dev.soylentnews.org/images/logo_vt100.png) no-repeat scroll 0 0
[07:58:23] <FatPhil> which is generated from: background: url("[% constants.imagedir %]/logo_vt100.png") no-repeat;
[08:01:37] <NCommander> janrinok|afk, if you pull up stories while in a nexus, you get THAT nexus's stories
[08:01:41] <NCommander> FatPhil, thanks
[08:02:07] <NCommander> FatPhil, though imgdir is pointed to the right place ...
[08:42:09] <crutchy> time to go to pipecode :p
[09:02:25] <FatPhil> there is something to be said for fresh starts. The problem is that the clean code-bases tend to get dirtier uglier and hackier over time, and you end up where you started.
[09:22:18] <crutchy> true
[09:23:23] <crutchy> i have no interest in perl though. probably never will
[09:24:27] <crutchy> so for me slashcode/rehash will always be dirty ugly hacks by others
[09:40:24] <janrinok|afk> NCommander: I'm not sure we are talking about the same thing. Once stories have been edited, but before they are released, I should be able to see all the stories that are waiting to go but require a 2nd ed to look at them, and I should be able to access all the stories that we have ever released. That was what was NOT showing - but is there currently so I'll keep my fingers crossed.
[09:52:18] <janrinok|afk> BUG - related to the reports I made above. When doing the second editor checks, if I open a story, complete my checks, and then save it, the story list only shows stories in the same nexus. The story list should show ALL stories unless it is intended to maintain a different list for each nexus. There is no way of selecting a different nexus from the story list page. Editing has therefore to stop at that point.
[09:57:33] <janrinok|afk> any devs still on here?
[09:58:38] <janrinok|afk> Without seeing the whole story list, I cannot set new stories to an appropriate time to release. At present, I can not edit any new stories.
[10:26:19] -!- Tachyon_ has quit [Read error: Connection reset by peer]
[10:26:40] -!- Tachyon [Tachyon!~Tachyon@xuco.me] has joined #dev
[11:00:57] -!- cmn32480 [cmn32480!~cmn32480@Soylent/Staff/Editor/cmn32480] has joined #dev
[11:14:41] <janrinok|afk> cmn32480: you still here?
[11:22:14] <paulej72> janrinok|afk: not a bug. When you are in a nexus you only see the nexus' stories. to see all stories go to the mainpage first then look at the stories list
[11:22:33] <janrinok|afk> how do you select 'all' on the story list page?
[11:22:41] <janrinok|afk> where is the selection?
[11:23:10] <cmn32480> I'm here Janrinok
[11:23:19] <janrinok|afk> are you able to edit cmn32480 ?
[11:23:36] janrinok|afk is now known as janrinok
[11:23:37] <cmn32480> no sir. Can't get logged in to stay
[11:23:58] <cmn32480> tryiong a different browser
[11:24:02] <cmn32480> it worked last night
[11:24:21] <janrinok> I'm logged in, but can only see 3 stories on the story page. Haven't a clue when the last story is due to go out, nor can I check for dupes or pick times for release
[11:24:30] <paulej72> janrinok: go to the mainpage first then to the stories list.
[11:24:36] <cmn32480> I saw that after I deleted my cookeis
[11:24:53] <paulej72> you are currently in soylentnews.or/meta
[11:25:18] <janrinok> so why do I have to keep going back the the front page?
[11:25:32] <janrinok> as editors, we always want to see the whole list
[11:26:04] <cmn32480> I can't get logged in.
[11:26:08] <cmn32480> neither can Coolhand
[11:26:21] <janrinok> paulej72: thx for the quick fix - but it will not work for long
[11:26:29] <paulej72> the whole list is only availabe from the main page insdie a nexus you only see nexus articles as a editory may only have privs to edit on a nuxus
[11:27:29] <paulej72> its not a bug its the way things work.
[11:27:32] <janrinok> there is something that needs tweaking in the logic there. Once in, I shouldn't have to keep re-proving who I am to see all the stories. For editing purposes, the nexuses are irrelevant
[11:27:58] <cmn32480> Janrinok
[11:28:02] <janrinok> go ahead
[11:28:04] <cmn32480> I can see the whole list/
[11:28:19] <cmn32480> I was able to log in under http not https
[11:28:28] <janrinok> I can - until I edit a story. Then it limits you to the nexus of that particular story and the list shrinks
[11:28:31] <cmn32480> last story goes out at 1358
[11:28:31] <paulej72> janrinok: you should not need to relogin to see the whole list.
[11:28:50] <cmn32480> paulej72 - every page change we are being asked to relogin
[11:29:04] <paulej72> use https not http
[11:29:10] <janrinok> why do I have to go the front page then? It is not something that we usually do. We work from the subs and the story list mainly
[11:29:16] <cmn32480> i can't even get logged in under https
[11:29:58] -!- CoolHand [CoolHand!~CoolHand@Soylent/Staff/Editor/CoolHand] has joined #dev
[11:30:04] <janrinok> hi ch
[11:30:08] <CoolHand> helo
[11:30:11] <paulej72> try deleting your cookies
[11:30:17] <cmn32480> I can log in under https. as soon as I change pages, I get logged out
[11:30:32] <CoolHand> I just deleted cookies and restarted browser and can't login
[11:30:56] <cmn32480> I have to press login at least twice to get logged in under http
[11:31:26] <cmn32480> but when I change pages, I get logged out.
[11:31:31] <CoolHand> I just opened a different browser and couldn't login
[11:31:43] <CoolHand> I got logged in twice earlier but got logged out right away
[11:31:59] <janrinok> my problems are small in comparison then :)
[11:32:13] <cmn32480> stay logged in and pack the queue Janrinok
[11:32:23] <cmn32480> and please delete my merge from last night. It was a test
[11:32:26] <janrinok> will do my best if it will let me
[11:32:33] <paulej72> the full story list is here: https://soylentnews.org
[11:32:33] <aqu4> ^ "3404 File Not Found"
[11:32:56] <paulej72> if you are not using that url you are in a nexus.
[11:33:12] <paulej72> I am not sure why we are seeing login errors
[11:33:57] <CoolHand> not actually seeing errors.. it just comes back from login attempt like I didn't enter anything
[11:34:03] <CoolHand> no error generated
[11:34:18] <cmn32480> back in 5
[11:34:24] cmn32480 is now known as cmn32480|brb
[11:37:31] <CoolHand> it seems to be working for me in lynx from my vps...
[11:38:13] <CoolHand> so that's something.. doesn't work in firefox/palemoon/ie from work, straight thru work connection or through sshuttle proxy thing thru my vps
[11:38:36] <janrinok> we area going to have to re-write the wiki a bit, but the workround suggested by paulej72 is a good short-term fix.
[11:40:14] <CoolHand> I went to /admin.pl and I got logged in, but then it logged out when I tried to view a different page
[11:40:17] <CoolHand> like cmn said
[11:41:25] <paulej72> really hard to fix as I can't see the same problem. Do you know if you have ipv6 service from your computer?
[11:42:48] <CoolHand> I don't believe so.. I think I do on my vps (which is working)
[11:43:14] <CoolHand> could be something to do with that...
[11:43:36] <CoolHand> I have ipv6 on my computer, but I don't think our network here at work routes it
[11:47:46] <paulej72> OK I removed the AAAA ipv6 record for soylentnews.org. It should be about 20 minutes for it to clear. This may help with logins
[11:49:05] <CoolHand> well, if it doesn't, you can add it back.. :)
[11:53:27] <janrinok> Enough stories in the queue to keep us going for another 6-8 hours or so, so sort out your login problems first
[11:53:49] janrinok is now known as janrinok|afk
[11:54:05] <CoolHand> it was working for about a minute on my laptop, now it's back to doing the same things.. :( never worked on my other machines though
[11:54:36] <CoolHand> (I know it hasn't been 20 minutes yet)
[12:06:12] <paulej72> if there is a problem with cookies, then NCommander will need to look at it. He was the last one to look at it.
[12:11:08] <cmn32480|brb> odd.. and possibly nothing, but I'm also not seeing the logo in the upper left corner. The link still works to get to the main page, but the picture isn't there
[12:12:13] <cmn32480|brb> browser is Chrome, v43.0.2357.81 m
[12:13:00] cmn32480|brb is now known as cmn32480
[12:13:08] <CoolHand> I think login issue is cookie issue. an AC posted the following and ncommander said he saw it also once (and it is what I'm seeing): One second ago (logged-in) I modded your post interesting, upon site-reload I was logged out (Cookie-content: "nobody" instead my Session-ID) and your Post above disappeared :)
[12:13:13] <paulej72> not seeing that issue either
[12:13:42] <paulej72> the logo issue that is or the cookie issue
[12:13:47] <CoolHand> I don't see logo either.. other's in in rehash thread reported not seeing logo..
[12:15:12] <CoolHand> are there still 2 front end servers running load balanced? could one have issue and not the other? (if not ipv6 which at this point I"m thinking unlikely)
[12:15:53] <paulej72> my thoughs are a db table is out of wack for logins
[12:18:24] <CoolHand> another user comment: Even clicking on "login" gets me immediately logged out again. My guess it has something to do with session storage server-side? Every 10th time or so I manage to get a SID (according to cookie), but as soon as I send a request, i.e. click on a link, I am "nobody" again, so my session cannot be picked up again.
[12:19:17] <CoolHand> since it's errtatic, it doesn't seem like db table to me, but since I have no familiarity with *this* database or setup really, my opinion isn't worth much.. :)
[12:22:06] <cmn32480> paulej72 - is this possibly due to the issues on hydrogen that you guys were talking about last night?
[12:22:39] <paulej72> hydrogen is out of service again for the most part
[12:22:52] <cmn32480> ok. Well never mind then.
[12:23:19] <paulej72> I think there are some db gremlins that snuck in druing the db migration
[12:23:51] <paulej72> need to get to work. check in soon
[12:24:06] <cmn32480> no problem. Thanks for all the hard work on this.
[12:45:54] -!- crutchy has quit [Quit: Leaving]
[12:58:05] -!- audioguy [audioguy!~audioguy@Soylent/Staff/Developer/audioguy] has joined #dev
[12:58:05] -!- mode/#dev [+v audioguy] by SkyNet
[13:26:10] janrinok|afk is now known as janrinok
[13:54:34] <janrinok> paulej72: just for info I have been unexpectedly logged out 1 time in the last hour
[13:54:58] <paulej72> restared rehash to fix css and plugin issue
[13:55:08] <janrinok> k, that might be it :)
[13:55:11] <paulej72> had to reload rehash
[13:55:39] <janrinok> it seems to be calming down a bit from this morning
[13:55:44] <paulej72> we had dev css files follow us from the copy
[13:55:58] <janrinok> ah, that wouldn't help
[13:58:59] <janrinok> we could do with an extra button on the admin menu for editors. 'Stories' should link all stories, and 'Nexus Stories' should display the stories that have the same nexus as the one being edited. Currently, editors don't care about nexuses, but if you edit anything other than main page you get taken away from the list that we usually work with.
[13:59:53] <janrinok> If I edit a meta story, I have to go back to front page to get to the story list that I want. Not very convenient.
[14:00:01] <paulej72> janrinok: I can do that
[14:00:28] <janrinok> when everything settles down - you probably have higher priorities at present
[14:03:00] <CoolHand> I'm getting a 503 when trying to hit: https://soylentnews.org
[14:03:00] <aqu4> ^ "3404 File Not Found"
[14:03:14] <janrinok> http(s)://soylentnews.org/hof.pl gives me a internal server error or misconfiguration
[14:03:31] <paulej72> CoolHand: I just rebooted try to log out and in again
[14:03:53] <CoolHand> well, I've haven't been able to login recently..
[14:04:54] <CoolHand> Error 503 Service Unavailable
[14:05:00] <CoolHand> from "Varnish cache server"
[14:05:12] <paulej72> yp seeing them now.
[14:06:07] <CoolHand> 503'ing on main site now..
[14:06:08] <FatPhil> 502 Bad Gateway
[14:06:40] <CoolHand> not main site is ok
[14:06:43] <janrinok> getting 500s, 503s for about 5 seconds
[14:06:45] <janrinok> guru meditation XID: 1486369671
[14:06:45] <CoolHand> now I mean
[14:07:32] <janrinok> admin.pl is OK. submission list 503
[14:07:32] <paulej72> not sure what the fuck is up now
[14:07:56] <CoolHand> every other refresh is 503'ing me on main site... load balancer?
[14:07:59] <janrinok> XID: 1486370189
[14:08:08] <CoolHand> (or one server in load balancer rather0
[14:08:09] <paulej72> just let me look please
[14:08:25] <FatPhil> I still have no logo, CSS still saying background: transparent url(//dev.soylentnews.org/images/logo_vt100.png) no-repeat scroll 0 0
[14:08:42] <CoolHand> main site is better for me now... and I have a logo now
[14:08:51] <janrinok> FatPhil: paulej72 is just pulling his hair out, the bald paulej72 will be back shortly
[14:09:08] <CoolHand> admin.pl is giving me a blank page instead of 503 now
[14:09:41] <CoolHand> (showing no html when viewing page source)
[14:10:04] <janrinok> CoolHand: I've got admin.pl but no submission list. Funny fault this
[14:10:38] <janrinok> I suspect there is a reboot in progress - I've been logged out again
[14:10:47] <CoolHand> I just got a forbidden 403 /index.pl on one computer and the other is funky
[14:11:18] <CoolHand> now my main page has come back on the one but is 'funky' too.. no graphics looks like lynx.. :)
[14:11:26] <CoolHand> now its back
[14:11:27] <janrinok> yep me too
[14:11:35] <CoolHand> back on both pcs
[14:11:55] <CoolHand> i have logo! I have login!
[14:11:57] <janrinok> everything back as it should be
[14:12:00] <CoolHand> and can change pages!
[14:12:28] <CoolHand> ..and I'm logged out..
[14:12:29] <CoolHand> :(
[14:12:38] <CoolHand> dang.. thought it fixed the login issue for a minute there
[14:12:51] <paulej72> try https login CoolHand
[14:13:00] <CoolHand> admin.pl is working again also
[14:13:06] <paulej72> and if logged out try logging in again
[14:14:00] <CoolHand> yeah, I can login at https://soylentnews.org (or main page), but when I go to another page I'm logged out
[14:14:00] <aqu4> ^ "3404 File Not Found"
[14:14:20] <janrinok> CoolHand: it has fallen out with you...
[14:14:57] <CoolHand> Ok, I guess that's it... I'll have to resign as editor and post as an AC from now on...
[14:17:26] <paulej72> CoolHand: try logging in now
[14:17:54] <CoolHand> paulej72: so far so good..
[14:18:13] <CoolHand> good on 2 computers...
[14:18:38] <paulej72> are you staying logged in?
[14:18:49] <CoolHand> yep.. doing several things.. all good so far
[14:18:51] <paulej72> if so I have a radical fix
[14:18:52] <CoolHand> seems "normal"
[14:18:58] <CoolHand> radical dude
[14:18:59] <CoolHand> :)
[14:19:14] <paulej72> everyone might need to log in again
[14:20:03] <CoolHand> I just tried posting comment and got "invalid form key"
[14:20:13] <CoolHand> ..and it logged me mout
[14:20:16] <CoolHand> out
[14:20:19] <paulej72> log in again. I killed the login keys
[14:20:26] <CoolHand> k
[14:20:26] <paulej72> all of them
[14:20:42] <CoolHand> yep, that worked
[14:20:48] <CoolHand> I think u r a hero, paulej72 !!
[14:20:54] <CoolHand> paulej72++
[14:20:54] <Bender> karma - paulej72: 3
[14:21:00] <CoolHand> gold star! :)
[14:22:42] <CoolHand> well, you're going to have to let us know what it was...
[14:23:17] <paulej72> NCommander: changed the logintokens to remove the location field. for some users if they had loginkeys already stored with a location they would interfere with the new ones
[14:23:19] * CoolHand is fixing to go to a big meeting 4 the next couple of hours, so glad I could finish up helping t-shoot/verify this
[14:24:12] <paulej72> janrinok: can you log in and stay logged in now?
[14:24:35] <CoolHand> I don't think janrinok had near as much issue as cmn32480 and I...
[14:24:49] <janrinok> I only drop out every now and then
[14:25:05] <janrinok> I'm in at the moment
[14:25:27] <CoolHand> did u reboot a server again.. had an internal server error again when clicking on story
[14:25:37] <paulej72> no CoolHand
[14:25:49] <CoolHand> ok, well, it's working now
[14:26:04] <CoolHand> I gots to go to my meeting.. good luck! things are starting to look up anyway :)
[14:33:50] <paulej72> FatPhil: you may need to do a forced reload to get the fixed css files
[15:33:55] janrinok is now known as janrinok|afk
[15:41:01] <cmn32480> logins seem to be working! way to go paulej72!
[15:44:05] <FatPhil> alles gutt
[15:44:26] <cmn32480> I'm sorry FatPhil, I don't speak jive.
[16:19:23] <CoolHand> The site seems really fast now.... :)
[16:22:31] <cmn32480> it is m,uch better Coolhand. and the logins work :-)
[16:22:43] <CoolHand> I know.. paulej72 is my hero!
[16:23:32] <cmn32480> him and ncommander both. you should have seen them last night.
[16:24:22] <cmn32480> they were rollin' through problems left, right, and sideways.
[16:24:31] <CoolHand> well, yah, ncommander obviously...
[16:24:33] <CoolHand> :)
[16:24:51] <CoolHand> I looked at a bit of logs from it.. crazy guys.. :)
[16:25:30] <cmn32480> it was really nifty to watch how somebody else handles it when things go catywompus
[16:26:55] <CoolHand> yep, always educational
[17:34:05] janrinok|afk is now known as janrinok
[18:54:07] -!- janrinok has quit [Quit: leaving]
[20:55:54] -!- TLA [TLA!~52027da6@bzd3-bxxi33-7-6-smos942.64-6.cable.virginm.net] has joined #dev
[21:18:13] -!- TLA [TLA!~52027da6@bzd3-bxxi33-7-6-smos942.64-6.cable.virginm.net] has parted #dev
[21:25:35] -!- cmn32480 has quit [Quit: See You Later]
[22:18:33] -!- crutchy [crutchy!~crutchy@709-27-2-01.cust.aussiebb.net] has joined #dev