Page 1 of 1

March 2024 Performance Issues

Posted: Thu Mar 07, 2024 7:20 am
by Matt K
Received some reports of sluggishness today. There was a memory leak in the database that I just fixed which should address the issue. Let me know here if anyone is sitll having issues and I'll take a closer look into it today.

Re: March 2024 Performance Issues

Posted: Thu Mar 07, 2024 7:26 am
by mgladdish
I literally got a CloudFlare error when trying to view this thread :biggrin:

Also been having intermittent errors connecting to MySQL over the course of the day.

Re: March 2024 Performance Issues

Posted: Thu Mar 07, 2024 7:29 am
by bitbckt
Much improved, particularly jumping to the latest posts on longer threads. Thanks!

Re: March 2024 Performance Issues

Posted: Thu Mar 07, 2024 7:34 am
by ghmerrill
Some of the "issue" may not be with this site. I was having very sluggish network performance (via our supposedly 40M DSL line) this morning for about half an hour and then it improved. But I'm currently getting half the download speed that we're supposed to -- 25Mbps vs. 40Mbps. How local that is, I haven't looked into -- because life is too short.

Re: March 2024 Performance Issues

Posted: Thu Mar 07, 2024 8:37 am
by BGuttman
I had noticed a bunch of duplicate posts (more than normal) today. Probably because the posters experienced the delay and thought it had not posted.

Note that if you double post and I catch it, I'll remove one. If one of the posts has more comment I'll keep that one.

Re: March 2024 Performance Issues

Posted: Thu Mar 07, 2024 8:47 am
by hyperbolica
It was bad, then it improved, and now it's bad again. I think it's larger than just this site. There was a cable break of a major line in the middle east. That may be causing traffic to re-route and disrupting us here.

Re: March 2024 Performance Issues

Posted: Thu Mar 07, 2024 9:06 am
by ghmerrill
There appear to be jagged spikes in "bandwidth outages" and DNS server issues beginning this morning. The graphs are saw-toothed. Up and down, up and down, ...

Re: March 2024 Performance Issues

Posted: Thu Mar 07, 2024 9:52 am
by Slidingmyway
BGuttman wrote: Thu Mar 07, 2024 8:37 am I had noticed a bunch of duplicate posts (more than normal) today. Probably because the posters experienced the delay and thought it had not posted.

Note that if you double post and I catch it, I'll remove one. If one of the posts has more comment I'll keep that one.
That happened to me. I accidentally posted the same thread 5 times. My apologies to whomever had to clean up my mess.

Re: March 2024 Performance Issues

Posted: Thu Mar 07, 2024 11:46 am
by Posaunus
Response speed briefly improved this morning (I think). Now again very slow. :frown:

Re: March 2024 Performance Issues

Posted: Thu Mar 07, 2024 2:11 pm
by EriKon
Yeah, takes quite long to load everything since one or two days. For me it's just on this page.

Re: March 2024 Performance Issues

Posted: Thu Mar 07, 2024 2:28 pm
by Posaunus
Just 5 minutes ago, Website speed magically improved. Almost immediate response! :good:

(Let's hope it remains so!)

Re: March 2024 Performance Issues

Posted: Thu Mar 07, 2024 2:30 pm
by EriKon
Yes, over here as well!

Re: March 2024 Performance Issues

Posted: Thu Mar 07, 2024 2:57 pm
by Matt K
Hmmmm interesting. I have a family issue today that cropped up so I haven’t had a chance to look into it. I think the connection stuff is definitely partially to blame. It may not have been a memory leak but if the connections are slow then it’ll leave handles to the DB open which would look like a memory leak.

It’s still a little slow for me as I type. I’ll keep my eye on it as my family stuff hopefully resolves today and see if I can (ideally) permanently fix tomorrow if there are any issues that are fixable by me

Re: March 2024 Performance Issues

Posted: Thu Mar 07, 2024 3:32 pm
by EriKon
Seems like it's not a constant thing over here. Sometimes it works super fast for a few minutes and then again it takes a minute to load everything (sometimes with error pop-up)

Re: March 2024 Performance Issues

Posted: Fri Mar 08, 2024 12:27 am
by mgladdish
Yeah, still very bad. Just got this when loading a topic:

Image

If I can be any help on the tech side, give me a shout. Techie creds: https://www.linkedin.com/in/mgladdish/

Re: March 2024 Performance Issues

Posted: Fri Mar 08, 2024 12:37 am
by Posaunus
STILL ... ... VERY ... ... SLOW

General Error
SQL ERROR [ mysqli ]

Lock wait timeout exceeded; try restarting transaction [1205]

An SQL error occurred while fetching this page. Please contact the Board Administrator if this problem persists.

Please notify the board administrator or webmaster: [email protected]

Re: March 2024 Performance Issues

Posted: Fri Mar 08, 2024 1:42 am
by slideandtraps
Experienced above SQL error this evening and personal total posts dropped to 1
with "800%" and "300%" of total posted in other topics.

Re: March 2024 Performance Issues

Posted: Fri Mar 08, 2024 6:59 am
by Matt K
Okay, think I found the culprit. Everything all good with my family too, one of my relatives had a minor surgery last night but all is good today. I'm going to keep my eye on it today and will need to put a more permanent fix in this weekend.

Re: March 2024 Performance Issues

Posted: Fri Mar 08, 2024 7:57 am
by EriKon
Seems to work fine now. Thanks Matt for taking care of that! Much appreciated!

Re: March 2024 Performance Issues

Posted: Fri Mar 08, 2024 8:34 am
by sacfxdx
Where's that AI when you need it? It should have fixed it automatically. I guess it's too busy faking nude pictures. :o

Good job Matt. Thanks for taking care of this.

Re: March 2024 Performance Issues

Posted: Fri Mar 08, 2024 9:04 am
by Matt K
sacfxdx wrote: Fri Mar 08, 2024 8:34 am Where's that AI when you need it? It should have fixed it automatically. I guess it's too busy faking nude pictures. :o

Good job Matt. Thanks for taking care of this.
Welllllll about that. Turns out the script I’ve been using to capture spam, bots, and AI has became quite expensive computationally because of some automatic rules compounded with an enormous amount of AI generated garbage. The solution is actually not a big deal, just didn’t expect to have it come to a head this quickly. I was hoping to hold off until the next LTS of Ubuntu came out and switch over to that and fix a bunch of stuff at the same time (next LTS drops in April)

Re: March 2024 Performance Issues

Posted: Fri Mar 08, 2024 9:36 am
by ghmerrill
Matt K wrote: Fri Mar 08, 2024 9:04 am ... enormous amount of AI generated garbage.
Dumb AI. :( Well (in virtue of my 10 years in a previous life as a commercial compiler writer), It takes a very well designed automated code generator (and read that as "often very expensive") to generate optimal code or anything approximating it. And if you're doing some or all of that on the fly at run time, then there's an added efficiency cost as well (in optimizing the optimizer). Other than that, you can only do your best with what you can get. :good:

Re: March 2024 Performance Issues

Posted: Fri Mar 08, 2024 9:52 am
by Posaunus
Matt,

All good this morning.

Thanks for your efforts on our behalf. :good:

Re: March 2024 Performance Issues

Posted: Sun Mar 10, 2024 2:43 pm
by Matt K
I think I have the spam busting script back in place more efficiently now. It just ran and spike performance for a few seconds but it shouldn't be much more noticeable than typical usage and runs every few minutes, but only if there have been posts. LMK if anyone has issues tonight/into tomorrow.

Re: March 2024 Performance Issues

Posted: Mon Mar 11, 2024 3:06 am
by ithinknot
Have you noticed that user post counts are displaying weird numbers? Sometimes it's just a bit wrong, in other cases longtime users are showing as 1...

At the time of writing this, mine says 1010 next to a post or when you open my profile, but if you click through to 'Search user's posts' it finds all 1173.

Must be indicative of something.

Re: March 2024 Performance Issues

Posted: Mon Mar 11, 2024 4:35 am
by Matt K
I reset one of the sites indexes the other day which has the side effect of recalculating the number of posts. I’m not sure why it thinks you have fewer posts than you do, unless the search results show duplicates for some reason

Re: March 2024 Performance Issues

Posted: Mon Mar 11, 2024 6:47 am
by harrisonreed
It might be counting quoted snippets as posts.

Re: March 2024 Performance Issues

Posted: Mon Mar 11, 2024 7:06 am
by ithinknot
harrisonreed wrote: Mon Mar 11, 2024 6:47 am It might be counting quoted snippets as posts.
Nah, it's just nonsense in some cases.

To pick a couple of posters in this thread:
memberlist.php?mode=viewprofile&u=9927
search.php?author_id=9927&sr=posts
56 vs 152

memberlist.php?mode=viewprofile&u=13350
search.php?author_id=13350&sr=posts
12 vs 32

Obviously this doesn't particularly matter in use; I'm just bringing it up in case it's diagnostically significant...