#soylent | Logs for 2024-04-03
« return
[00:20:18] -!- Runaway92 [Runaway92!~Runaway@45.250.opi.ysr] has joined #soylent
[00:22:12] <Runaway92> SN used to be a bastion of free speech; now it is a bastard of free speech. Unfortunately, that was the only thing it had going for it, and now that that is gone, so is everyone else.
[00:25:16] -!- Runaway92 has quit [Quit: Client closed]
[04:19:45] <chromas> "but in spite of the fact that this is the case"
[04:20:04] <chromas> I keep seeing that in articles.
[04:21:24] <chromas> Looks like it's always Digital Information World
[04:22:35] <chromas> https://feed.zeroindex.org
[04:22:36] <systemd> ^ 03feed
[05:28:32] -!- lld_ [lld_!~lld@2001:f40:qgg::yqp] has joined #soylent
[05:30:25] -!- lld has quit [Quit: Reconnecting]
[05:30:49] lld_ is now known as lld
[05:31:31] -!- lld has quit [Client Quit]
[05:31:41] -!- lld [lld!~lld@2001:f40:qgg::yqp] has joined #soylent
[05:35:25] -!- lld has quit [Client Quit]
[05:35:35] -!- lld [lld!~lld@2001:f40:qgg::yqp] has joined #soylent
[06:26:13] -!- lld has quit [Ping timeout: 252 seconds]
[06:26:24] -!- lld [lld!~lld@2001:f40:qgg::yqp] has joined #soylent
[07:27:41] -!- _dx3bydt3 [_dx3bydt3!~|dx3bydt3@129.224.kiz.sl] has joined #soylent
[07:31:07] -!- dx3bydt3 has quit [Ping timeout: 252 seconds]
[07:31:28] -!- dx3bydt3 [dx3bydt3!~|dx3bydt3@129.224.qpo.uzq] has joined #soylent
[07:34:25] -!- _dx3bydt3 has quit [Ping timeout: 252 seconds]
[07:39:14] <janrinok> for a minute there I thought that ari was actually going to be arguing in support of Runaway....
[07:40:34] <chromas> Remember that one time Zumi and TMB were arguing, then at some point she realized she was in agreement with him the whole time?
[07:42:41] -!- halibut has quit [Quit: Timeout]
[07:53:10] -!- halibut [halibut!~halibut@CanHazVHOST/halibut] has joined #soylent
[08:08:02] <janrinok> lol - vaguely
[09:05:38] -!- aristarchus [aristarchus!~aristarch@103.219.rjg.vos] has joined #soylent
[09:07:26] <aristarchus> "That one time"? We now have hard evidence that chromas is an AI, and hallucinating. Never happened. Revisionist Soylentil. What do the Pauls say?
[09:27:50] -!- aristarchus has quit [Quit: Client closed]
[11:10:48] -!- Runaway1956 [Runaway1956!~OldGuy@the.abyss.stares.back] has joined #soylent
[11:13:19] -!- Runaway has quit [Ping timeout: 252 seconds]
[13:49:27] -!- lilitsyunetsi has quit [Quit: Gateway shutdown]
[15:23:01] -!- lilitsyunetsi [lilitsyunetsi!~lilitsyun@37.252.sp.xyt] has joined #soylent
[15:32:13] <prg> Are there issues with the site again? I'm now getting a timeout instead of a 500.
[15:34:01] <janrinok> It is working for me
[15:34:48] <prg> Now it's working for me again, too.
[15:35:24] <janrinok> We were getting brief timeouts yesterday but we think it is external to our site. Perhaps linode/akamai are doing something.
[15:36:12] <janrinok> By 'we' I mean fab23 and myself couldn't pin it down to anything else :)
[15:36:42] <janrinok> *fab23 and I
[15:36:59] <janrinok> how long did the timeout last?
[15:37:12] <schestowitz[TR]> site loads ok from here
[15:37:27] <prg> Tried it I think three times.
[15:37:38] <prg> Timed out after half a minute or so.
[15:38:19] <janrinok> Sorry, I can not think of anything that we can do at this end. Perhaps a traceroute will show a problem for you?
[15:40:58] <prg> traceroute gets me to a23-203-147-39.deploy.static.akamaitechnologies.com, after that only * * *
[15:41:27] <prg> It seems to be working intermittently.
[15:41:52] <janrinok> yeah, it looks like the same problem that we saw yesterday.
[15:42:40] <janrinok> I don't know enough to know if this indicates a problem at akamai or a problem with our server talking to akamai.
[15:43:07] <prg> Oh well, looks like I'll just have to get back to work then. Thanks for looking into it.
[15:43:28] <janrinok> lol - you can always keep watching the traceroute :)
[15:44:00] <prg> watch -n 10 traceroute ...
[15:44:24] <prg> Maybe it changes after a while.
[15:46:32] <janrinok> optimist....
[16:06:01] <drussell> It's up and down via IPv6... See all the notes in #staff
[16:06:24] <drussell> Works fine on IPv4, but 6 is very flakey. I verified it here, sometimes you can ping,sometimes not...
[16:06:31] <drussell> Sometimes the site responds, sometimes not.
[16:07:31] <drussell> I thought maybe it was just ping packets, but I checked it more thoroughly yesterday between two laptops, one on v4 one on v6, the IPv6 connectivity is definitely flakey right now.
[16:07:50] <drussell> (HTTPS connectivity is also up and down, I mean... on IPv6)
[16:08:02] <drussell> not just pings
[16:09:33] <prg> I'm using IPv4 though.
[16:10:00] <drussell> Perhaps it's getting worse then... There are some IPv4 errors the past few days in #staff also, but it's been fine here every time I tested it on v4
[16:10:33] <drussell> IPv6 pings have been up and down more rapidly than the tides the past couple days, I've been meaning to ask if anyone knewwhat was going on...
[16:10:37] <fab23> the problems we see since yesterday are mostly IPv6, to check also 'mtr www.soylentnews.org' is more helpful, as it clearly shows packer loss over time. :)
[16:11:12] <drussell> I wsn't sure if it was just sylvester's connection, but no, it tracks here also on the Shaw connection I'm testing from.
[16:11:45] <fab23> depending on OS / version traceroute may be used as traceroute6 to check IPv6, mtr has a -4 or -6 option to change default of the prevered protocol from the OS.
[16:12:08] <drussell> even windows you can do a ping -6 or ping -4 to force it to either protocol
[16:13:05] <drussell> (and tracert -4 or -6)
[16:13:51] <fab23> if ping / traceroute are different binaries or have -4 / -6 option depends on OS and version of OS :)
[16:14:14] <drussell> indeed. I always keep typing ping6 when I need 'ping -6' haha
[16:14:30] <fab23> e.g. on macOS 13.x (Ventura) it still is ping/ping6 and traceroute/traceroute6
[16:14:32] <drussell> (and vice versa)
[16:15:12] <drussell> yeah, FreeBSD it is ping6
[16:15:38] <fab23> on FreeBSD 13.2 both ping/ping6 are there, but ping -6 also works :)
[16:16:54] <drussell> on 12.x on my servers it doesn't have -6... My laptops run 14, but I wasn't checking from there. :)
[16:17:38] <drussell> I still always just reflexively seem to type ping6
[16:21:37] <fab23> IPv6 still is quite new, around since 20+ years :)
[16:22:18] <requerdanos> I have no public IPv6 address, and ping -6 something.or.other gives me "ping: connect: Network is unreachable"
[16:22:26] <requerdanos> so, yeah, too new for me I guess.
[16:25:14] <fab23> more like "not existing"
[16:28:39] <Ingar> arch has -4 -6
[16:31:48] <Ingar> site works fine here
[16:33:34] <requerdanos> sylvester is having a heck of a time for the past few days
[16:44:12] <drussell> Indeed. My point was basically that "it's not JUST sylvester," it's borked from here also.
[16:44:33] <drussell> Up and down... Up and down. Perhaps the site is seasick? ;)
[16:44:57] <Ingar> it needs some viagra to stay up
[17:01:57] <schestowitz[TR]> hard to debug site availability when you outsource traffic or pass it around
[17:02:18] <janrinok> Can you say for certain whether it is our site or whether if might be something akamai might be working on?
[17:02:57] <fab23> what is ugly that it is only the IP of www but not the mail host.
[17:03:24] <schestowitz[TR]> does it route to cogentco.com?
[17:03:29] <schestowitz[TR]> based on traceroute it is
[17:03:37] <schestowitz[TR]> somebody mentioned akamai
[17:03:47] <janrinok> I wonder if that is k0lie's system?
[17:04:25] <janrinok> we are supposedly using akamai/linode as far as I know. Unless somebody is trying to switch it over to something else.
[17:05:04] <schestowitz[TR]> 8 be3393.ccr31.bos01.atlas.cogentco.com (154.54.47.141) 86.031 ms be2870.ccr41.lon13.atlas.cogentco.com (154.54.58.173) 13.136 ms be3393.ccr31.bos01.
[17:05:04] <schestowitz[TR]> 9 be2101.ccr32.bos01.atlas.cogentco.com (154.54.82.38) 83.988 ms be3599.ccr21.alb02.atlas.cogentco.com (66.28.4.237) 83.990 ms be2101.ccr32.bos01.atl
[17:05:04] <schestowitz[TR]> 10 be3599.ccr21.alb02.atlas.cogentco.com (66.28.4.237) 83.014 ms be2878.ccr21.cle04.atlas.cogentco.com (154.54.26.129) 102.169 ms be3599.ccr21.alb02.a
[17:05:04] <schestowitz[TR]> 11 be2717.ccr41.ord01.atlas.cogentco.com (154.54.6.221) 98.471 ms be2878.ccr21.cle04.atlas.cogentco.com (154.54.26.129) 101.224 ms be2717.ccr41.ord01.
[17:05:07] <schestowitz[TR]> 12 be2717.ccr41.ord01.atlas.cogentco.com (154.54.6.221) 98.702 ms 98.953 ms be2831.ccr21.mci01.atlas.cogentco.com (154.54.42.165) 110.840 ms
[17:05:10] <schestowitz[TR]> 13 be2831.ccr21.mci01.atlas.cogentco.com (154.54.42.165) 111.895 ms be2432.ccr31.dfw01.atlas.cogentco.com (154.54.3.133) 120.222 ms 120.484 ms
[17:05:13] <schestowitz[TR]> 14 be2433.ccr32.dfw01.atlas.cogentco.com (154.54.3.213) 120.427 ms 132.618 ms be4463.rcr51.dfw06.atlas.cogentco.com (154.54.94.58) 129.325 ms
[17:05:16] <schestowitz[TR]> 15 be3905.nr61.b044570-0.dfw06.atlas.cogentco.com (154.24.6.70) 130.929 ms 130.396 ms 129.811 ms
[17:05:18] <schestowitz[TR]> 16 * be3905.nr61.b044570-0.dfw06.atlas.cogentco.com (154.24.6.70) 129.132 ms 130.365 ms
[17:05:18] <fab23> nooooooooooo
[17:05:20] <schestowitz[TR]> 17 * * *
[17:05:58] <fab23> schestowitz[TR]: it is only IPv6 which causes trouble, based on my Nagios
[17:06:09] <schestowitz[TR]> sorry for longpaste
[17:06:20] <schestowitz[TR]> souunds like isp issue
[17:06:23] <schestowitz[TR]> or cdn
[17:06:42] <fab23> as far as I know there is no CDN in use for SN
[17:07:33] <fab23> and as mention, the IPv6 for soylentnews.org and mail.soylentnews.org are in the same /64 subnet
[17:09:48] <janrinok> I get connected to akamai - no cognetco seen at all
[17:09:58] <janrinok> *cogentco
[17:10:32] <schestowitz[TR]> https://www.akamaistatus.com
[17:10:33] <systemd> ^ 03Akamai Status
[17:10:57] <schestowitz[TR]> Apr 3, 2024
[17:10:57] <schestowitz[TR]> Unresolved incident: Edge Delivery Issues in Dallas and Atlanta.
[17:11:15] <janrinok> yep, I saw that yesterday
[17:12:10] <schestowitz[TR]> i worked for clients with this outsourced CDN BS for over a decade
[17:12:29] <fab23> as far as I know Linode is just owned by Akamai, but I guess their infra is still different
[17:12:31] <schestowitz[TR]> in most cases they ccause more trouble, even down time, than they are "worth"
[17:13:10] <schestowitz[TR]> whe they are down your site if "down" even when it is night
[17:13:10] <schestowitz[TR]> and debugging technical issues is harder due to external factors outside your control
[17:13:10] <schestowitz[TR]> aside from privacy issues among others
[17:14:40] <schestowitz[TR]> <techrights-sec2> I reach a dead-end with traceoute starting at Akamai
[17:14:40] <schestowitz[TR]> [18:14] <techrights-sec2> There is a weird loop justoff the end of the LAN, a loop which includes
[17:14:40] <schestowitz[TR]> [18:14] <techrights-sec2> some unidentified nodes (labelled with stars)
[17:15:40] <fab23> traceroute / mtr / ping may show * if the host in the route does not answer to this packets.
[17:17:08] <janrinok> I appear to be losing most at the Amsterdam edge
[17:18:16] <schestowitz[TR]> can you not toggle off akamai cdn/proxy in linode ui?
[17:18:31] <drussell> We're not using any proxy or cdn.
[17:18:34] <schestowitz[TR]> seems like the problem is some company trying tro be superclebver like clownflare
[17:19:08] <janrinok> it all comes to a stop at 2600:3c00::f03c:91ff:fe98:90b LINODE-US (NET6-2600-3C00-1)
[17:19:09] <schestowitz[TR]> maybe the webhost is owned by on\e :-)
[17:19:16] <drussell> Traceroute goes straight through to 2600:3c00::f03c:91ff:fe98:90b (the SN server)
[17:19:49] <drussell> 2600:3c00::f03c:91ff:fe98:90b is soylentnews.org
[17:19:49] <janrinok> OK that ties in
[17:21:22] <drussell> I go Shaw (a bunch on non-reverse lookup addresses) -> seattle.Level3 -> akamai akamai akamai -> a bunch on non-reverse addresses -> SN server
[17:21:41] <drussell> It is 18 hops in total from this laptop.
[17:22:54] <janrinok> 17 hops for me via Amsterdam and Washington. over 80% loss in Amsterdam
[17:24:03] <janrinok> Those non-reverse addys are all Linode
[17:24:38] <janrinok> ... I'm still not sure what this tells me :)
[17:25:07] <drussell> It is 18 hops via IPv4 from here as well, ending with 23.239.29.31
[17:25:32] <drussell> Well, it is quite possibly nothing to do with SN's server... There may be other, broader issues.
[17:25:34] <drussell> Russians?
[17:26:19] <schestowitz[TR]> occam's razor
[17:26:27] <schestowitz[TR]> sites get lazy about ddos
[17:26:31] <schestowitz[TR]> outsourced to cdn
[17:26:35] <schestowitz[TR]> and isps try to save costs
[17:26:40] <schestowitz[TR]> by over-caching
[17:26:49] <schestowitz[TR]> then the admin wastes time
[17:27:00] <schestowitz[TR]> and the site becomes unavailable - worse than under ddos
[17:31:36] <janrinok> well I think that there are bigger names than ours complaining about this issue and at least they are aware of it. I just hope that they get it fixed!
[17:33:07] <schestowitz[TR]> a good host will ACK the issue
[17:33:10] <schestowitz[TR]> at their end
[17:33:16] <schestowitz[TR]> to avoid the client time--wasting
[17:33:44] <schestowitz[TR]> but in MBA era? not a chance
[17:33:57] <schestowitz[TR]> they try to not get sued and look like 100% uptime to outsiders
[18:04:35] <schestowitz[TR]> traceroute geminiprotocol.net
[18:04:44] <schestowitz[TR]> 9 be3382.rcr21.rtm01.atlas.cogentco.com (154.54.39.218) 20.746 ms 18.428 ms 16.432 ms
[18:04:50] <schestowitz[TR]> https://geminiprotocol.net and https://gemini.circumlunar.space are offline :/
[18:04:57] <schestowitz[TR]> seems like other hosts impacted
[18:04:59] <schestowitz[TR]> same isp
[19:07:32] <chromas> must be the gemini cdn
[19:14:06] -!- x0908d3b2 [x0908d3b2!~quassel@2605:ea00:1:j::qvto:oymn] has joined #soylent
[19:52:28] -!- x0908d3b2 has quit [Quit: http://quassel-irc.org - Chatee cmodamente donde sea.]
[19:52:58] -!- x0908d3b2 [x0908d3b2!~quassel@2605:ea00:1:j::qvto:oymn] has joined #soylent
[20:11:31] -!- esainane has quit [Remote host closed the connection]
[21:14:27] <AlwaysNever> Quote: Runaway: I've been banned for a month - the fact that I argue the ban, prompts staff to add additional months. // Nice shit. What's up? I would like just to ask: What is the new sacred cow which cannot be blasphemed about?
[21:15:28] <AlwaysNever> so that I know how to pretend to toe the line THEY are drawing
[21:20:57] <Ingar> did the site get compromised ?
[21:21:39] <Ingar> can't seem to login in
[21:23:36] <Ingar> oh wait I'm stupid
[21:30:11] <AlwaysNever> Ingar: Putin dit it.
[22:06:37] <chromas> inb4 "sacred cow" is offensive Indophobic colonizer terminology
[22:51:47] <AlwaysNever> chromas: I don't have guilt before which to kneel
[22:56:36] -!- anubi [anubi!~anubi@2607:fb91:a94:pxlp:rzgs:lomk:xuqr:wpin] has joined #soylent
[23:00:43] -!- anubi has quit [Client Quit]
[23:48:50] -!- NotSanguine has quit [Quit: NotSanguine]