#soylent | Logs for 2025-10-05
« return
[00:44:43] -!- progo83 [progo83!~progo@eegc-73-589-96-43.nwrknj.fios.verizon.net] has joined #soylent
[01:00:12] -!- bender has quit [Remote host closed the connection]
[01:00:22] -!- bender [bender!bot@Soylent/Bot/Bender] has joined #soylent
[01:00:52] -!- devbot has quit [Remote host closed the connection]
[01:02:04] -!- Loggie [Loggie!Loggie@Soylent/BotArmy] has joined #soylent
[01:08:51] -!- c0lo [c0lo!~c0lo@124.190.mg.jlq] has joined #soylent
[01:11:37] -!- mrpg [mrpg!~mrpg@Soylent/Staff/Editor/mrpg] has joined #soylent
[01:11:37] -!- mode/#soylent [+v mrpg] by Imogen
[01:19:42] <mrpg> submit=https://www.reuters.com/business/database-startup-supabase-notches-5-billion-valuation-funding-round-2025-10-03/
[01:34:23] <mrpg> submit=https://www.reuters.com/business/database-startup-supabase-notches-5-billion-valuation-funding-round-2025-10-03/
[01:34:37] <mrpg> Guru meditation
[01:35:02] <mrpg> It's meditating.
[01:38:42] <c0lo> =sub by mrpg https://www.reuters.com
[01:43:26] -!- mrpg has quit [Quit: mrpg]
[01:56:41] <c0lo> systemdead is deadbeef
[02:00:12] <halibut> =submit https://www.reuters.com by mrpg
[02:00:37] <halibut> Seemed unlikely that would do it, but no harm in trying.
[02:00:55] <halibut> I had forgotten about this. It is over 10 years old at this point, but still works:
[02:00:59] <halibut> traceroute -m 128 bad.horse
[02:01:26] <halibut> (The certificate chain for signed.bad.horse has a similar easter egg.)
[02:06:21] <ted-ious> systemd can't accept your submission when he's not here. :)
[02:08:33] <halibut> Forgot to check that. Of course, systemd is always there ... waiting to crash when you least expect it. I once shouted gibberish while standing on a windy cliff, and systemd submitted an article for me.
[02:09:05] <ted-ious> I don't believe you.
[02:09:14] <halibut> ... and then overwrote my diary with a bunch of binary that can only be parsed with journalct.
[02:09:25] <halibut> (journalctl)
[02:09:29] <ted-ious> Ok now I believe you.
[04:06:58] <chromas> oh yeah the internet burp earlier
[04:27:38] -!- systemd [systemd!~systemd@pid1] has joined #soylent
[04:28:23] <chromas> I should bring back my xml journald replacement
[04:34:00] <c0lo> =sub by mrpg https://www.reuters.com
[04:34:01] <systemd> ✓ Sub-ccess! "03Reuters.com" (2p) -> https://soylentnews.org
[04:37:25] <c0lo> =sub https://www.msn.com
[04:37:31] <systemd> ✓ Sub-ccess! "03MSN" (1p) -> https://soylentnews.org
[04:39:48] <c0lo> =sub https://economictimes.indiatimes.com
[04:39:50] <systemd> ✓ Sub-ccess! "03Database Startup Supabase Notches $5 Billion Valuation in Funding Round - the Economic Times" (2p) -> https://soylentnews.org
[04:40:24] <c0lo> I give up.
[05:22:52] <chromas> https://www.youtube.com
[05:22:55] <systemd> ^ 03Yello - Oh Yeah (Official Video)
[06:45:34] <c0lo> https://moneywise.com
[06:45:37] <systemd> ^ 03Las Vegas tourism plunge leaves casino workers ‘on call’ and struggling for shifts
[06:53:47] <ted-ious> https://www.youtube.com
[06:53:49] <systemd> ^ 03Oh Yeah!
[06:54:48] <chromas> Do we have one of those tiny violins to play for the people of Vegas?
[07:05:58] <ted-ious> Oh it has vocals too! https://www.youtube.com
[07:05:59] <systemd> ^ 03360° (Oh Yeah?) - Propellerheads (Ft. De La Soul)
[07:45:14] <chromas> rappists
[07:45:56] <ted-ious> That's some amazing headphones audio.
[07:48:53] <chromas> I only hear left and right
[07:53:04] <chromas> Ian Roberts, American hero
[08:04:08] -!- Runaway has quit [Ping timeout: 245 seconds]
[08:04:42] <Ingar> Error 503 Backend fetch failed
[08:07:22] <Ingar> and it doesn't recover
[08:07:47] <Ingar> oh it does... just very slowly
[08:17:53] -!- c0lo has quit [Ping timeout: 245 seconds]
[09:19:17] <fab23> maybe the site needs an Espresso as I do :)
[10:30:56] -!- c0lo [c0lo!~c0lo@124.190.mg.jlq] has joined #soylent
[10:51:01] <c0lo> https://www.youtube.com
[10:51:04] <systemd> ^ 03She taped everything that aired on tv for 40 years to fight the rise of false news #movies #tv
[11:19:56] <donaldo> "She was right" says the youtuber ( https://www.youtube.com ). What has happened that make her right?
[12:25:29] <donaldo> https://www.youtube.com
[12:25:32] <systemd> ^ 03Why Are Software Engineers Quitting Microservices?
[13:13:55] -!- donaldo [donaldo!~donaldo@315.38.1.669.dynamic.jazztel.es] has parted #soylent
[13:41:22] <c0lo> https://www.youtube.com
[13:41:25] <systemd> ^ 03Brits think American Christianity is weird. here's why.
[13:56:34] -!- donaldo [donaldo!~donaldo@315.38.1.669.dynamic.jazztel.es] has joined #soylent
[13:58:52] <donaldo> I don't know it it's just me, othe registered nick is not working properly - I authenticate successfully (apparently) to NickServ, but my registered nick is not used when I post...
[13:59:07] <donaldo> *or the registered
[14:04:45] <donaldo> let me try again...
[14:05:16] <donaldo> nope
[14:05:29] <donaldo> the bot says: "NickServ!NickServ@ircservices.sylnt.us - You are already logged in as AlwaysNever."
[14:05:32] <donaldo> but nope
[14:08:12] <fab23> donaldo: did you had another nick name when you connected to the IRC network up until you did log in with nickserv and then did a /nick donaldo (e.g. change your nick name)?
[14:20:34] <donaldo> I connect to the IRC server from a Bash shell where the system user is "donaldo", perhaps I need to tell the IRC client explicitely which nick I want even before registering with NickSrv? I will try that.
[14:22:59] -!- donaldo [donaldo!~donaldo@315.38.1.669.dynamic.jazztel.es] has parted #soylent
[14:24:20] -!- donaldo [donaldo!~donaldo@315.38.1.669.dynamic.jazztel.es] has joined #soylent
[14:24:24] <donaldo> Hello again
[14:24:26] <donaldo> nope
[14:25:35] <fab23> I have no idea what your irc client is doing :( But for sure something ugly.
[14:25:58] <donaldo> meanwhile, I share this fine music with your: https://www.youtube.com
[14:26:00] <systemd> ^ 03You Ghost My Heart
[14:26:11] <donaldo> I hope it's not AI-generated...
[14:32:15] -!- donaldo has quit [Quit: Time wasted on IRC: 8 minutes 34 seconds]
[14:33:45] -!- AlwaysNever [AlwaysNever!~AlwaysNev@315.38.1.669.dynamic.jazztel.es] has joined #soylent
[14:34:03] <AlwaysNever> Ok, now it works
[14:34:49] <AlwaysNever> So I have to tell the IRC client which nick to use, it's not enought to successfully authenticate to NickServ
[14:35:40] -!- AlwaysNever has quit [Client Quit]
[14:40:31] -!- donaldo [donaldo!~donaldo@315.38.1.669.dynamic.jazztel.es] has joined #soylent
[14:40:36] -!- donaldo [donaldo!~donaldo@315.38.1.669.dynamic.jazztel.es] has parted #soylent
[15:03:03] -!- AlwaysNever [AlwaysNever!~donaldo@315.38.1.669.dynamic.jazztel.es] has joined #soylent
[15:36:29] <AlwaysNever> it seems to be working now
[15:47:52] <fab23> \o/
[16:39:16] <kolie> going to cycle all services, 5min unavailability incoming.
[16:39:34] -!- c0lo has quit [Remote host closed the connection]
[16:41:00] -!- Loggie [Loggie!Loggie@Soylent/BotArmy] has joined #soylent
[16:41:02] -!- x0908d3b2 [x0908d3b2!~quassel@2605:ea00:1:j::qvto:oymn] has joined #soylent
[16:41:04] -!- jje [jje!jje@ddjffrbpjin.info] has joined #soylent
[16:41:06] -!- Deucalion [Deucalion!~Fluff@188.226.nuy.w] has joined #soylent
[16:41:06] -!- Deucalion has quit [Changing host]
[16:41:06] -!- Deucalion [Deucalion!~Fluff@Soylent/Staff/IRC/juggs] has joined #soylent
[16:41:06] -!- mode/#soylent [+v Deucalion] by Imogen
[16:41:12] -!- fab23 [fab23!fabian@ysxbxfyv.wenks.ch] has joined #soylent
[16:41:14] -!- Bytram [Bytram!~Bytram@fd0f:9ae5:lvhi::xr] has joined #soylent
[16:41:14] -!- Bytram has quit [Changing host]
[16:41:14] -!- Bytram [Bytram!~Bytram@Soylent/Staff/Developer/martyb] has joined #soylent
[16:41:14] -!- mode/#soylent [+v Bytram] by Imogen
[16:41:15] -!- Ingar [Ingar!~ingar@51.15.nj.ri] has joined #soylent
[16:41:16] -!- progo83 [progo83!~progo@eegc-73-589-96-43.nwrknj.fios.verizon.net] has joined #soylent
[16:41:51] -!- [R] [[R]!~rms@45.44.m.yt] has joined #soylent
[16:41:53] -!- prg [prg!~prg@ryfhuvb.de] has joined #soylent
[16:41:55] -!- Xyem [Xyem!~xyem@2a01:7e00::znpv:pygj:pvjy:kivh] has joined #soylent
[16:41:55] -!- Xyem has quit [Changing host]
[16:41:55] -!- Xyem [Xyem!~xyem@Soylent/Staff/Developer/Xyem] has joined #soylent
[16:41:55] -!- mode/#soylent [+v Xyem] by Imogen
[16:41:57] -!- chromas [chromas!~chromas@Soylent/Staph/Infector/chromas] has joined #soylent
[16:41:57] -!- mode/#soylent [+v chromas] by Imogen
[16:42:03] -!- SSLbits [SSLbits!~SSLbits@CanHazVHOST/SSLbits] has joined #soylent
[16:42:17] -!- Fnord666 [Fnord666!~Fnord666@fd0f:9ae5:lvhi::xr] has joined #soylent
[16:42:18] -!- Fnord666 has quit [Changing host]
[16:42:18] -!- Fnord666 [Fnord666!~Fnord666@Soylent/Staff/Editor/Fnord666] has joined #soylent
[16:42:18] -!- mode/#soylent [+v Fnord666] by Imogen
[16:42:46] -!- janrinok [janrinok!~janrinok@fd0f:9ae5:lvhi::xr] has joined #soylent
[16:42:47] -!- janrinok has quit [Changing host]
[16:42:47] -!- janrinok [janrinok!~janrinok@Soylent/Staff/Editor/janrinok] has joined #soylent
[16:42:47] -!- mode/#soylent [+v janrinok] by Imogen
[16:43:47] -!- kolie2 [kolie2!~not_kolie@fd0f:9ae5:lvhi::xr] has joined #soylent
[16:44:42] -!- inz [inz!~inz@wbi.fi] has joined #soylent
[16:52:59] -!- Loggie [Loggie!Loggie@Soylent/BotArmy] has joined #soylent
[16:52:59] -!- mode/#soylent [+nt #soylent] by irc.soylentnews.org
[16:52:59] -!- mode/#soylent [-o Loggie] by atheme.soylentnews.org
[16:52:59] -irc.soylentnews.org:#soylent- *** Notice -- TS for #soylent changed from 1759683178 to 1392328183
[16:52:59] -!- mode/#soylent [+P #soylent] by atheme.soylentnews.org
[16:52:59] -!- Imogen [Imogen!Imogen@imogen.dusty.attic] has joined #soylent
[16:52:59] -!- mode/#soylent [+o Imogen] by atheme.soylentnews.org
[16:52:59] Imogen changed topic of #soylent to: SN Main Channel | Keep discussions civil | https://soylentnews.org | Impersonating another user's nick is forbidden | Some PISG charts: https://stats.sylnt.us | This channel IS logged and publicly displayed here https://logs.sylnt.us
[16:52:59] -!- devbot [devbot!devbot@Soylent/BotArmy] has joined #soylent
[16:53:01] -!- bender [bender!bot@Soylent/BotArmy] has joined #soylent
[16:53:01] -!- bender has quit [Changing host]
[16:53:01] -!- bender [bender!bot@Soylent/Bot/Bender] has joined #soylent
[16:53:02] -!- inz [inz!~inz@wbi.fi] has joined #soylent
[16:53:03] -!- [R] [[R]!~rms@45.44.m.yt] has joined #soylent
[16:53:03] -!- Ingar [Ingar!~ingar@2001:bc8:1640:mwzr:pxpv:wq:jjgg:moil] has joined #soylent
[16:53:05] -!- fab23 [fab23!~fabian@2001:8a8:izvs:s::i] has joined #soylent
[16:53:08] -!- x0908d3b2 [x0908d3b2!~quassel@2605:ea00:1:j::qvto:oymn] has joined #soylent
[16:53:08] -!- esainane [esainane!esainane@sendjocq.space] has joined #soylent
[16:53:08] -!- progo83 [progo83!~progo@eegc-73-589-96-43.nwrknj.fios.verizon.net] has joined #soylent
[16:53:12] -!- prg [prg!~prg@ryfhuvb.de] has joined #soylent
[16:53:15] -!- Bytram [Bytram!~Bytram@fd0f:9ae5:glhn::h] has joined #soylent
[16:53:15] -!- Bytram has quit [Changing host]
[16:53:15] -!- Bytram [Bytram!~Bytram@Soylent/Staff/Developer/martyb] has joined #soylent
[16:53:15] -!- mode/#soylent [+v Bytram] by Imogen
[16:53:35] -!- jje [jje!jje@ddjffrbpjin.info] has joined #soylent
[16:54:06] -!- Xyem [Xyem!~xyem@2a01:7e00::znpv:pygj:pvjy:kivh] has joined #soylent
[16:54:06] -!- Xyem has quit [Changing host]
[16:54:06] -!- Xyem [Xyem!~xyem@Soylent/Staff/Developer/Xyem] has joined #soylent
[16:54:06] -!- mode/#soylent [+v Xyem] by Imogen
[16:54:16] -!- Fnord666 [Fnord666!~Fnord666@fd0f:9ae5:glhn::h] has joined #soylent
[16:54:18] -!- Fnord666 has quit [Changing host]
[16:54:18] -!- Fnord666 [Fnord666!~Fnord666@Soylent/Staff/Editor/Fnord666] has joined #soylent
[16:54:18] -!- mode/#soylent [+v Fnord666] by Imogen
[16:54:50] -!- janrinok [janrinok!~janrinok@fd0f:9ae5:glhn::h] has joined #soylent
[16:54:53] -!- janrinok has quit [Changing host]
[16:54:55] -!- janrinok [janrinok!~janrinok@Soylent/Staff/Editor/janrinok] has joined #soylent
[16:54:58] -!- mode/#soylent [+v janrinok] by Imogen
[16:55:11] -!- chromas [chromas!~chromas@Soylent/Staph/Infector/chromas] has joined #soylent
[16:55:13] -!- mode/#soylent [+v chromas] by Imogen
[16:55:21] -!- SSLbits [SSLbits!~SSLbits@CanHazVHOST/SSLbits] has joined #soylent
[16:55:36] -!- Deucalion [Deucalion!~Fluff@188.226.nuy.w] has joined #soylent
[16:55:36] -!- Deucalion has quit [Changing host]
[16:55:36] -!- Deucalion [Deucalion!~Fluff@Soylent/Staff/IRC/juggs] has joined #soylent
[16:55:36] -!- mode/#soylent [+v Deucalion] by Imogen
[16:55:46] -!- kolie2 [kolie2!~not_kolie@fd0f:9ae5:glhn::h] has joined #soylent
[16:57:47] -!- kolie2 has quit [Changing host]
[16:57:47] -!- kolie2 [kolie2!~not_kolie@Soylent/Staff/Management/kolie] has joined #soylent
[16:57:47] -!- mode/#soylent [+o kolie2] by Imogen
[16:58:32] kolie2 is now known as kolie
[17:00:01] <kolie> Ok I put in additional tweaks on the VM which should most likely eliminate 99% of the issues identified.
[17:02:55] <kolie> numbers look better across the board, we are running a backup rn and i dont see a slow down
[17:04:55] <janrinok> Excellent - sorry about the restart, I didn't even know you were online
[17:05:43] <kolie> You can see in the proxmox logs I reset it like 30 seconds before you logged in.
[17:05:52] <kolie> All good, it caused a DB burp.
[17:05:57] <kolie> its fine now.
[17:06:32] <janrinok> All I was seeing were 404s and the site had disappeared
[17:06:33] <kolie> im not sure if all services are operational though probably just bounce them all to be sure
[17:06:53] <kolie> i was cycling them when the second reset hit.
[17:06:59] <janrinok> lol
[17:07:12] <kolie> and with alwaysnever - logging into nickserv - has nothing to do with your nick name in the server
[17:07:20] <kolie> ./nick anything sets the name
[17:07:36] <kolie> logging into nickserv registers your connection as a registered user - but doesnt set the display nick name
[17:07:39] <janrinok> proxmox is looking much healthier
[17:08:07] <kolie> janrinok, what stats are you comparing to?
[17:08:20] <janrinok> with regards to what?
[17:08:27] <kolie> you said it looks healthier
[17:08:55] <janrinok> we've got more RAM, CPU usage is down, htop looks good
[17:09:32] <kolie> yea the ram gets paged in and out over time - what I see is
[17:09:38] <kolie> sometimes we are 100 ram, but 0 swap usage
[17:09:44] <kolie> which means we are using ram very efficiently
[17:09:50] -!- halibut [halibut!~halibut@CanHazVHOST/halibut] has joined #soylent
[17:09:59] <kolie> the time were its 100 ram and swap i/o is super high
[17:10:05] <kolie> thats when the cpu load gets to like 200x
[17:10:21] <kolie> the command "vmstat 1" shows the swap in/out
[17:11:40] <janrinok> where is that command?
[17:11:45] <kolie> Anyways, based on the notes you've provided, my own digging, and watching the numbers and trends over the past 3 days since thursday, friday, and saturday, I put in a few targetted fixes and made some tweaks to the config
[17:12:06] <kolie> SSH to soylent-host-0 ( 102 ) and just use "vmstat 1" on the command line.
[17:12:13] <kolie> devops@soylent-host-0:~/irpg$ vmstat 1
[17:12:13] <kolie> procs -----------memory---------- ---swap-- -----io---- -system-- -------cpu-------
[17:12:13] <kolie> r b swpd free buff cache si so bi bo in cs us sy id wa st gu
[17:12:13] <kolie> 5 0 0 26322104 395896 8095220 0 0 13758 4474 20193 15 17 6 71 6 0 0
[17:12:13] <kolie> 4 0 0 26313224 395896 8095372 0 0 64 4945 38456 42163 31 13 55 0 0 0
[17:12:14] <kolie> 4 0 0 26317320 395896 8095476 0 0 64 600 25385 23106 18 9 73 0 0 0
[17:12:33] <janrinok> ah, OK, I was searching in Proxmox....
[17:13:50] <janrinok> It appears at this early stage that your fixes have worked
[17:14:53] <janrinok> I will leave my software running for the moment. Currently, I am getting request responses in around 200-400 ms
[17:16:22] <janrinok> Most delays over the last 24 hours have been around 5-15 seconds, with 503 being generated at the 30 second mark.
[17:16:25] <kolie> Yea the VMs pretty beefy it would be like a 300-400$ plan on digitalocean or similar.
[17:19:20] <janrinok> looks like the backup completed a few minutes back
[17:22:35] <janrinok> Are you happy if I update the front page story to FIXED or would you rather I hang on a short while longer?
[17:22:51] <janrinok> Don't count chickens ... etc
[17:25:37] <kolie> I think we already were on the right track, and thats why we had some improvement, just now stuttering.
[17:25:49] <kolie> And with this its even more addressed.
[17:25:57] <kolie> We had basically, 500 before
[17:26:06] <kolie> killed the 500s, but got weird loads.
[17:26:11] <kolie> plus some 500s
[17:26:17] <kolie> Now its probably good.
[17:26:39] <kolie> theres one issue still that idk how im going to approach it.
[17:26:59] <janrinok> I'll leave the front page story as is for the moment.
[17:27:01] <kolie> ima hit you up in the other place
[17:27:27] -!- ted-ious [ted-ious!~tedious@ted.ious] has joined #soylent
[18:07:46] <Ingar> ftest
[19:07:22] -!- Runaway1956 [Runaway1956!~OldGuy@the.abyss.stares.back] has joined #soylent
[19:08:19] <Runaway1956> Oh, God, family gatherings are killers. I stuffed myself, and was told to take more home so it wouldn't go to waste.
[19:08:40] <Runaway1956> Home now, I want so badly to indulge some more - but I'm STUFFED!
[19:09:35] <Runaway1956> I had one of the ten best steaks of my life today, maybe top five. It's so hard to decide when I can't compare side-by-side.
[19:54:58] -!- c0lo [c0lo!~c0lo@124.190.mg.jlq] has joined #soylent
[22:56:20] <c0lo> https://www.youtube.com