When you have thousands of tasks viewable in a Kanboard board view, you run into some performance issues. This post is meant to summarize my findings and highlight a potential way forward.
Until now, people have mostly told me things along the lines of “Just use MySQL, SQLite can’t possibly handle that kind of stuff”, which… doesn’t seem like an entirely accurate depiction.
- My testing (further details below) says otherwise, so far.
- Hey, if my computer has local apps with SQLite databases that are a hundred times bigger than my 1400 tasks test case and those work fine, why couldn’t Kanboard?
Besides, I can’t migrate my existing kanboard instances to MySQL because, as I pointed out in the this topic, the SQLite to MySQL migration usecase isn’t really well documented or supported (there is no official migration script, and the third party script I could find out there doesn’t handle accents and special characters), so “SQLite to MySQL” leaves me in a dead end.
Hence I pushed towards SQLite again to see if it was really impossible for it to scale. I created a synthetic test case by taking the first 10 “Inferno” canti of Dante’s Divine Comedy, converting them to Kanboard-compatible .CSV format where each verse ends up being imported as an individual task by Kanboard, and each canto is a column, so you end up with 1368 tasks spread across 10 columns. It looks like this (and this is in compact mode, otherwise in full card height mode the image would be over 17 thousand pixels high):
I hope you appreciate the ridiculous over-the-top approach I took to stress-testing this issue. 
Notes:
- To rule out any potential shared web hosting latency issues from the testing equation, and because I wanted to see SQLite’s “best performance”, I’m running my test instance on a local web server, which is simply a laptop with a cheap SSD. That will be fast enough for 99% of cases.
- You will need (for now at least) to use Rafael Camargo’s workaround to break past the 999 tasks limitation
…and you know what? Performance turned out to be just fine, and actually pretty fast except one thing: when you have thousands of tasks in the “Board” view in those 10 columns, it takes 6 seconds between the time you release the mouse after moving a task, and the time the move is fully recorded (i.e. the spinning progress indicator stops). But the problem doesn’t happen if you’re only showing a subset of the tasks (e.x.: with a search) or if you’re in individual task editing mode. Only in a heavily loaded board view. So this hints that database isn’t the issue, SQLite isn’t the main performance culprit (or maybe even not a culprit at all) according to my tests. But I only tested switching the various views (Summary, Board and List view), moving tasks around in board view and editing tasks in their individual editing view… did I forget anything important to test here?
Anyhow, so far my findings are consistent with this bug report where a fellow nicknamed “ChangePlaces” found out that the board view’s Javascript was causing the slowdowns. Unfortunately, that person, who had already identified the causes of the problems and had a patch in the works (which was reportedly solving the problem on his side), never shared the code… and when I tried to encourage that person to release their work for the public to either merge or improve, Frédéric closed the issue ticket and said to bring that to the forums, cutting off the communication that was beginning to happen. Is that really better? 
Now we’re splitting the info in two places, one of which he (as far as I know) mostly ignores*. Urgh.
So here I am. This is a forum post that summarizes my findings, in the hope of raising awareness of this issue and the potential fix, and to encourage code-savy folks here to look at it or to convince that “ChangePlaces” person (who isn’t on the forum) to release their code experiments.
*: I understand a volunteer project maintainer’s personal need to reduce the amount of noise and fuzzy requests they get—I really do—especially as a project becomes more popular… but “Go to /dev/null !”—especially for a case where it’s a clearly defined performance bug—sounds counterproductive to me. Now I’ve spent two hours preparing & writing this forum post, re-contextualizing and re-explaining problems again, and basically duplicating all of this in the vague hope that “someone” takes it to heart, and this kind of situation—which isn’t the first occurrence—feels to me like we can’t have issue tickets that are “too active” or they will be shut down. I know it’s Free Software™ and if I’m not paying Frédéric I’m not allowed to complain, but damnit, for someone who also spends his free (and unpaid) time trying to improve the software (with testing and feedback, rather than code) I have to say that this behavior is a bit frustrating and at times demoralizing, to be honest. I like to think that my contributions aren’t worthless; otherwise, if I didn’t care and believe in Kanboard’s potential to improve for everybody, I would probably be using Trello and move on with life.