Bad bug importing a large library of markdown files

[StygianRivers]

Version, Operating system
Cherrytree version 1.2.0
Cherrytree version 1.6.3
Ubuntu 25.10

(I first somehow got my hands on 1.2.0 and got this bug. Uninstalled it from my software centre, deleted any leftover files I could find, and installed the latest stable version instead. I still got the exact same bug.)

Describe the bug
Around 90% (at a glance) of all my files' content is missing after importing it into cherrytree. I checked the original files, and they're complete and healthy, but the content doesn't show up in cherrytree. When I enable the word count I can see only about 10% of the words there should be per file (I did not check every file). When I then scroll through, I see that it's not simply cut off after a certain point in the file, but even around the start there are entire sentences missing.

As an example, at the very start of one particular file it should say: There is something missing. Something has been taken away. It's an all-consuming sensation, destroying every coherent thought that could follow in its wake.

It's been midnight for twenty-one minutes exactly, and it's only now that the adrenaline kicks into gear, struggling along long-overgrown pathways.

While in Cherrytree it says: There is something missing. Something has been taken away. It's an all

It's been midnight for twenty-one minutes exactly, and it's only now that the adrenaline kicks into gear, struggling along long-overgrown pathways.

To Reproduce

1. Clean install of cherrytree 1.2.0 or 1.6.3.
2. Import 'Folder of Markdown files' where the folder has various subfolders and branches. Mine had about 1400 .md files total.
3. Enable word count.

[innovationvectors]

"the adrenaline kicks into gear, struggling along long-overgrown pathways." .. I can relate with that sentiment.

I do not understand why such a large volume of files needs to imported. It is asking for failure. What are you trying to achieve? If it is some book there are other ways. But there might be some rogue markdown files causing havoc. My approach would be not to place them in one large file but treat them as a network or a NoSQL database such as eXistdb.org. Cherr y Tree *.ctd file are XML. But to continue with your quest split the 1400 files into folders and try injecting the folder at a time. Or even a script which runs through your entire repertoire importing one at a time, auto checking for errors and then the next one. Another aproached is to validate your external files for errors using similar scripting tactics. A markdown validator might be automated by a script.. Now I have that volume of CherryTree files *.ctd dotted around my desktop. The tool I use to manage them is Recoll. You can use Recoll to index the folder containing 1400 files and hunt for errors. I can expand on this approach which works for me with CherryTree used extensively in my workflows. But never do I import 1400. Postscript.

I found this which is ideal for your case.
https://github.com/Final-State-Press/markdown-validator

You could even modify that script to call CherryTree by command line to import AFTER each validation. But I would rather go for several CherryTree files containing less than 1400. Prhaps 14 CherryTree Containers (*.ctd - not *.ctb) each containing 100 files as an example.

[StygianRivers]

Thanks for the detailed response!

I understand that maybe that's an unusual amount of files, as I'm a writer. I got too excited when I heard about Cherrytree's "search all notes" function as well as the ability to code formatting. Those 1400 files (actually more like 1100, another application told me later? So maybe the wrong number is part of the error?) are my entire library of notes and writing content.

To be clear, they're already in a large branched network of folders and sub-folders, so I can't easily split them into even numbers of files. I could try importing several layers deep of sub-folders, but I don't know if I can honestly be bothered to do that when I don't even know how deep I'll have to go.

Thanks for your response, either way.

[innovationvectors]

I am into a forest of branches so I know the problem of order out of chaos
As a writer have you tried Scrivener which is meant for authors?
There is no need to split into even numbers of files. I just gave an example. The could be one file in one folder and 100 files in another folder.

What I would do is make an index of the paths to various folders. The the sc ipt can rip through them, You can even automate grabbing that tree structure depending on your knowledge. Is this Ubuntu or Windows and I might offer tips. Thee must be a root folder somewhere to start traverse the tree. All files with extension *.md.
In fact I use a bunch of tools. With that volume you really need to try Recoll as suggested. And if you have them in folders you can use RipGrep in command line .. very fast.. The key is finding some context inside the files rather than the file names.

Explain if you are in Windows, Ubuntu or Mac sinc the advice varies.

[StygianRivers]

I'm using Ubuntu 25.10 and Cherrytree 1.6.3.

I tried Scrivener a while ago and hated it, I'm now (since a couple days ago) trying Joplin and really liking it, actually—it even imported the same file structure without error—but I figured it was rude to talk about really liking a similar program while criticizing another. I definitely don't mean to be rude.

I'll try out importing the 8 subfolders in my root folder separately and let you know how that goes, the next time I get behind a computer. (Though there's one folder that holds maybe 50 other folders with in total, at a guess, half the data of the entire thing, so I fear for its life.) I appreciate the replies.

[innovationvectors]

CherryTree is a great tool and I integrate extensively into my workflow. Which incidentally is like an author's work but describing technology in metaphorical terms. I write various processes witth illustrations embedded. In Sublime Text (which I urge you to try) these are "snippets". I have more than 1000 snippets scattered thrugh my desktop and I urge you to try Recoll to get a birds eye view of your assets. Also in Ubuntu install Albert and you can write your first snippet by three keys into Albert .. then just type "snip add". You are in.

But for your immediate task I suggest that you explore pandoc (it is also seen in the Subl Text Build System.

https://pandoc.org/MANUAL.html

You can have a script pick up your md files and pass through pandoc, gradually build a document from snippets. Cycling through your folders. I assume that you have some scheme to index through your files, even by date?

Other tools to explore are Scribus for superior styling of books. Your 1400 snippets can be placed into textframes by automation script. And images. Also look at Scribus Generator for customising books. There is Obsidian which I don't use myself yet. So there you have it. A Swiss Army Knife approach. But the bottom line is I suspect that you have some validation error screwing up your import. My advice is start with pandoc. Then progress from there. Also another line. Consider quarto which is an advanced form of markdown. Ext *.qmd.

[StygianRivers]

So I tried importing the files folder by folder, starting with the small folders, and very quickly (2 folders and 20 total markdown files in) I found the same or similar bug, with it cutting off a large part of the contents. So most likely there's something wrong with my files and how they interact with Cherrytree.

I tried looking into the markdown validator, but I'm too new to coding to understand how to configure that effectively, and honestly, I can't be bothered to spend time learning. It may just be that Cherrytree is not for me(being unwilling to put in the effort). I feel bad saying it, because I really do appreciate all your tips and suggestions. That hasn't changed. Thanks for trying.

I'm also brand new to Github. What's the etiquette? Should I close this issue, if I'm not willing to work on it further?

[innovationvectors]

I'm sorry that I could not convince you to at least try Sublime Text. Then we can sing from same hymnsheet. I 'll mke it easy for you. Here is the link. https://www.sublimehq.com/ I would stay away fro GitHb until you have cracked the markdown puzzle. And again pandoc is a great tool for authors. There is another editor Zettlr for markdown,
After that just let this wither on the vine with a closing note.

[innovationvectors]

Insight .. I notice from first post that this glitch occurs here .. "all**-**consuming" .. so perhaps the character "-" is causing an issue. Just a hunch.

[StygianRivers]

I had that thought for a second too, that's why I included the second paragraph in the example: that one has several dashes that show up fine. Because of that, I think the first one is just coincidence. I later also noticed that content was cutting off in places with no special characters.

[innovationvectors]

Try some sample lorem ipsum content files .. for a test to see if it is content. Spawn lorem ipsum files.