Your Hot New Stuff Here

KNewStuff, our Framework designed to make it as easy as possible for people to Get Hot New Stuff for whatever app they're running, and for the people who make those apps to add that same functionality to the apps, has for some time suggested that the KDE Store has the ability to accept uploading of content directly from the application. Unfortunately, due to the way this was originally built (which was essentially built around the idea that People Are Nice) turned out to be a fairly effective vector for spam and malicious content.

The result of this is that for several years, while KNewStuff has had an implementation of the OCS Content Create functionality, there is no serverside implementation which supports it as it stands. Which is a shame, especially when, while we may not be able to do it through the API because people are, apparently, sometimes Not Nice, what we can do instead is make it easier for people who are nice to learn how to add their own stuff to those listings.

Quickly Now!

How about that discoverability, eh?

 

Enter NewStuff.UploadPage and a few little hooks scattered through the rest of the Framework, such as a context action in NewStuff.Page which pushes an UploadPage to the page stack, and a reimplemented KNS3::UploadDialog which wraps up the QtQuick based UploadPage in a dialog, so those who use the existing dialog don't end up without functionality. This was added from the merge request right over here, if you would like to see what the process looked like.

How about using a Dialog? Even More Discoverable-er!

Some Side Effects

While working on this innocent seeming bit of stuff, i ran into few little things that needed to be done first. Of course, i could have hacked this and just Made It Work, but also that is not what we do in KDE.

The first, probably smallest thing is that this inherently shows up in our mobile apps, because it's now a part of both Page and Dialog now. Not that there's a problem with people finding their way to the store and uploading things there, but if we catch just a couple of people who would have otherwise not uploaded their stuff for the rest of the world to see, well, i'll chalk that down as a win :D

As for more technical things, the way in which one would get a list of the providers known to a KNewStuffCore::Engine was previously all manner of awkward, and really does not fit into how a modern Qt application works (that is, you'd call one function to get some string IDs, then you'd call another function to get each provider, and there was no good way to keep track of what might happen if another got added late, and so on). So, we now have a little model, KNSCore::ProvidersModel, which handles this, and just gives you a list of all the Providers known to an Engine, with a bunch of roles that lets you read all the information you need.

Did you know KNewStuff supports multiple providers?

Another long standing issue in KNewStuffCore::Engine, as non-critical as it was, is that it never properly understood what to do with a relative path for a knsrc file. In daily operation this isn't an issue at all, since you just pass it the filename of an installed one, or the full path if you manage your own, and that works just fine. For testing, though, being able to pass in a relative path to for example the multi-provider test knsrc shown in the upload page screenshot above is just kind of handy.

KNewStuff is supposed to be able to handle multiple languages. This turns out to never have worked right for the StaticXML provider, which would simply run through all the title entries, and assign all of them to the provider's title string, which would invariably end up with the last one. Now, though, it will pick the correct one, depending on the current system locale.

Thoughts?

The intention (of course) for NewStuff.UploadPage and the functionality its name suggests is to be expanded in the future. We are looking into ways for how to add the upload functionality back in, in a way that does not also turn into a spam vector. One method we have considered here is to, basically, retain the functionality in OCS as it exists right now, but have a vetting process for users (which then will need some way to essentially "knock" to be allowed API upload privileges). Another idea we have floated is to not allow new content to be added, but for it to be possible to change existing content. I am not really sure precisely what to do, and i also see the two previous ideas could potentially work simultaneously. But, for the time being, while we work this out, we now have a way to at least guide people through to how to actually add their own content to the store (and, indeed, any other source that KNewStuff handles).

---

The word of the day is: Incremental. Because this obviously isn't the end product here, there will be website-less uploads again, sometime, somehow!

Graceful Downtime

Occasionally, services need to go down for some reason or another. That's just a thing that happens. Servers need software updates too, but what happens to the clients that need those services? Well, luckily we've got these HTTP Status responses, and one of those is the 503 response, which is described as meaning "Service Unavailable". The actual English interpretation of that is more akin to temporarily unavailable, and even better, if it is also paired with a "Retry-After" header, with a time sometime in the near-ish future, then we can reasonably assume that that what's happening is an expected downtime, or maintenance.

As of 5.84, KNewStuff will handle that on the various services it supports, and in particular the KDE Store (since that is accessed through the Attica library, which also got a bit of attention to allow for this introspection to be done).

What if error, but helpful?!

It's a small thing, but it means a lot to our users that they don't just sit there staring at something and then get a "lol it's broken" type message, when in fact we do know better what is going on. A side effect of putting in this bit of infrastructure in the library also means we now have somewhere to deal with other unfortunate networking situations in a more graceful manner - not done yet, of course, but we've got a starting point for it.

Just a little one this time, but i thought it was worth mentioning none the less. Not everything has to be huge or groundbreaking, sometimes "this little thing is nice for our users" is also good ;)

---

The word of the day is: Failure. Because that's something we have to deal with sometimes :)

KDE's Author Avatars and Not Phoning Home

For a long time now, KDE's about dialogues (the ones you find in the help menus in many of our apps) have supported the ability to fetch a bunch of information about the authors of our applications from the KDE Store through the Open Collaboration Services API. It does this by sending a request to the store for each of the authors who have an ocs username defined, to get the information on user avatars, profile links, and a few other potentially interesting details.

Phoning Home

If you are paying attention, you are now squinting at the screen and going "wait, I saw that title up there, and it clearly says not phoning home, and this is phoning home". You are, of course, entirely correct, and while it certainly was never done with ill intent, it is hard to dispute the fact that it is, in fact, phoning home. As a result, we have been poking about with ways of fixing this, without dropping overly much of the functionality.

Some of those crazy author people what make Discover happen

You might now be thinking what functionality, and sure, it's seems like perhaps a small thing, but remember: KDE is a world-wide community of people who work on this massive pile of software, and the about dialogue is basically the first place where many people are likely to learn of that concept, if they decide to go and look. As such, we want to put our best face forward, and in this case what we want is to firstly not be causing any undue internet traffic (because nobody wants that), but also we want to literally put faces forward.

Not Phoning Home

As of a couple of weeks ago, the KDE Store has had a simple url which will give you a redirect to the avatar that you have set on there, at the scale you need of it, and that's what we're using now. You can use it yourself as well - there's a short introduction to it over here. You can also link directly to your profile in a similar manner, by just linking to store.kde.org/u/someusername - for example, mine is store.kde.org/u/leinir.

Oh look, authors what do money managementy things!

The result is two patches, which landed recently: One for Kirigami, and one for KXMLGui. All of this was originally spawned from a desire to reduce the complexity of KXMLGui itself, and specifically removing the requirement on Attica, which while certainly not heavy, was still doing an inordinate amount of work for an arguably small gain in this particular instance.

There is more to be done on this, mind you - if you read comments on the Kirigami patch, you will notice that one intention is to expand this avatarness to support things other than the store as well, and while i do have some ideas on the topic, if you've got some, don't hesitate to give me a poke! (not "why don't you just use gravatar", that's not the point here, any expansion of this needs to be generic, otherwise it doesn't make a huge deal of sense - the reason we're doing it this way is in part that the ocsUsername field is a part of KAboutData already, and it will Just Work(TM) for anybody who already uses that data structure).

Over To You?

Part of this work could arguably have been done before (specifically that not phoning home part, which from a user perspective is a checkbox which defaults to being off), but since this was originally based on removing the Attica requirement, frankly, this particular angle is something which i must confess, sadly, to have quite entirely missed.

And that, i guess, is the point here: It is staggeringly obvious in hindsight that this was totally a case of phoning home, and while it was never intended to be that, and to my knowledge was never used like that, it took someone pointing at it and going "yeah, this is doing a phoney homey thing" for me to go "yup, it clearly is, let's make that not happen". So that's what you can do here: Be that person that points out a blindingly obvious case of phoning home to someone who's just not noticed in a decade that that's what's happening, furthering KDE's goal of ensuring our software operates in a privacy conscious manner.

Finally, a direct question: Having to tick those boxes in each about dialogue is, arguably, a little far the other direction again, and having that tick apply to all about dialogues in our software would make that bit of life just a tiny bit simpler. Not a lot, arguably, but a little is also good, of course. The question, then, is this: Is it an acceptable tradeoff in privacy terms to let this tick box affect the about boxes in all our software, or should it be per-application? Personally i'm leaning toward it being acceptable, but i wanted to put it to the wider community before diving into that bit of the code.

---

The word of the day is: Care. Because we should, and we do :)

Peruse 2.0 Beta 1 "The Long Awaited Beta"

A fair while ago, in the before times of late 2016, a release was made of a piece of software known as Peruse. Since then, it spent some time getting work done on it on and off, until sometime last year when we decided that it really was time to stop the thing just floundering in some free software equivalence of development hell, and actually get it ready for its next release.

Peruse?

First things first. For those of you who are new, Peruse is KDE's comic book reader project, which consists of the reader application, Peruse Reader, and the comic book creation tool called Peruse Creator.

The project's releases further include a library by the name of libacbf, which allows applications to use Advanced Comic Book Format (ACBF) metadata without doing manual parsing or generation of the xml data. If this doesn't mean anything to you, don't worry, as a reader or maker of comic books, what that really means is that you get rich comic books that are not simply a collection of images in a specific order.

What's New?

Since that most recent release, a fair lot of stuff has happened, including a great deal of polish and fixing in various places throughout the project. In the following, i am going to try and highlight the more impactful ones, in a rough order of coderiness.

Frame Based Navigation

One frame of a much larger page, so you can see all the gorgeous details

From the very start of Peruse, one of the goals was to eventually be able to read comics not only on a page-by-page basis, but more akin to what you might do if you get really deeply into something on paper and focus hard on each panel, except in a guided manner. This is one of the things that ACBF allows the authors of comic books (using an application like Peruse Creator) to do, by letting them mark out areas on a page, and assign them an order, which then lets applications like Peruse Reader know precisely what to show and in what order. Not only is this useful in a way that makes comic books more engrossing to read, but it also makes it much easier to do so on a smaller, hand held device like a phone.

Translated Comics

Translations - because language shouldn't be a barrier to enjoying a good comic book

One of the powerful features of ACBF is the concept of TextLayers. What those are, in essence, is translations for comic books - a way of putting formatted text on top of an image, for speech bubbles and the like. Peruse now has support for this, and if you would like, there's a blog entry right over here talking about that more deeply.

Better Navigation in Books

The sidebar pulled out, with page thumbnails

 Another feature in ACBF is the ability to link between various parts of a book to another. It seems simple enough on the surface, but this allows for some really nifty stuff. On a basic level, you can create a table of contents and jump from there to other places, but you can also create things akin to a choose your own adventure book using this method. There is plenty more to say about this, which is why there is a whole blog entry over here talking about that specifically.

Greatly Expanded Creator

Editing a text-area with a link in

 

The version of Peruse Creator shipped with Peruse 1.2 was an extremely simple tool, designed primarily to just create a simple comic book archive and fill it with pictures, and annotate it with some basic metadata. In Peruse 2.0, while it is not yet able to do everything that ACBF will do, a great deal of it is (such as, but not limited to, references, embedded files, as well as the three types of page areas: frames, jumps, and text areas). In short, Peruse Creator is starting to become a tool that is actually, genuinely good for creating comic book archives and annotating them with ACBF metadata.

Collection Cache

Previously, Peruse would scan the system on startup every time, and build a full database of all the books. This, as you might expect, was an extraordinarily heavy operation, and with any more than a few tens of books, it effectively made it impossible to use Peruse in the casual and immersive fashion the rest of the application intends.

Graduating from Playground

A more organisational thing that happened was that Peruse has graduated from being a Playground project (KDE's experimental section for projects that are not considered quite full featured yet), and is now a proper, full member of KDE. For most people this is a small detail, but one of the larger effects of this is that it means that KDE considers it a "real" project, and it gets attention for things like translations.

ACBF Library

The details of a book

 

Another kind of user we envision for Peruse is other people also working on software which handles comic books, and particularly ACBF enhanced ones. Peruse's ACBF support was always intended to be usable by others (it was built specifically to only require QtCore, but annotated with enough bits to also use it from QML), and as of Peruse 2.0 we are releasing the library versioned according to the Peruse release. It was also expanded greatly, and now supports almost the entirety of the ACBF specification. Round-tripping documents is still a work in progress, but it's already pretty solid.

More on this in a later blog post, but for now, the cmake module is called AdvancedComicBookFormat, and your starting point is AdvancedComicBookFormat::Document in the AcbfDocument.h header.

Kirigami 2

The original release of Peruse was built on top of Kirigami 1, during the early days of the development towards what would eventually become Plasma Mobile. One of the first things to happen after the release of version 1.2 was to port Peruse to Kirigami 2, and the result for the user is partly just more modern and stable code, but also much more easily navigable using a keyboard. Since then, the Peruse team has been working to bring more of the features of Kirigami' which didn't exist in the first version into Peruse, such as the way the search field works, the way scrolling pages are handled, page row layers, action handling, and a whole bunch more.

Gentler, Safer Threading

The way images are served into QML is something called an Image Provider. The old method used by Peruse was a set of simple image providers, which attempted to throttle their own behaviour somewhat, but in reality tended to block operations fairly badly, and also caused severe threading overload on slower systems (such as, for example, on a PinePhone). The end result of that was that you would regularly end up not being given an image when you loaded a book, or your thumbnails would take forever to load.

The new code has been refactored heavily, and now uses what is called an asynchronous image provider, and also makes sure to not ever launch more threads than are actually available on the system it's running on. To you as a user, what this means is not only a more reliable image generation, but often also faster, and safer to boot (which is not really a result of this change to a new kind of image provider, so much as the old code just not being very stable).

No More Submodules

The thing in this list which will be least relevant to someone who just wishes to learn about what Peruse does for reading comic books, this one is for the packagers and self-builders: Peruse previously used an external bit of code to read rar based comic book archives (the ones with the file suffix cbr, and seemingly the most common out in the wild). This turned out to cause endless headaches for packagers, and the decision was made to import that code fully into Peruse.

Gimme!

Before we get to the downloads: This is a beta version, and you should expect it to behave like one of those: Things may well be a bit broken or unpolished, and we will be very happy to see reports any bugs you run into over on the Peruse product category on bugs.kde.org. With that out of the way, head over to peruse.kde.org to grab yourself a shiny new copy of Peruse :)

---

The word of the day is: Polish. Because that's something to be done when you've got a beta out of the door ;)

Advanced Comic Book Linking

In my previous post about Peruse's support for ACBF Textareas, i talked about how the formatting system there allowed for all manner of niftiness with languages, and rotating text, and making it coloured, and styled, and all of that visual fanciness. In this one, i'm going to talk a bit more in depth about a quite powerful feature the Textareas have that i kind of glossed over in that one: Hyperlinks.

Yes, hyperlinks. Links, like the one you just clicked on above to work out what that last post actually said about textareas (thanks for reading! :) ). Except that these links are inside any of the things in an ACBF document which can hold paragraphs of text, being primarily Textareas, References, and Annotations. The reason this is a separate post (apart from the fact the other one had a different focus) is that ACBF recently gained the ability to add a resource target (that is to say, a href property) to the Jump element.

Linking to some URL, and protecting people

For a simple starter, the most usual suspects for hyperlinks work for this: Websites (http(s)://) and email links (mailto:), primarily. If you click on one of those in Peruse, don't worry, we're not going to send you anywhere without making sure you want to actually go there. On the web, you usually would be expected to trust whatever you're clicking on, but inside a book, you'd reasonably expect to be constrained, so before sending you anywhere that isn't inside the book, you get a cute little popup that shows you the address of the link you clicked on, and asks if you really do want to follow it.

Additionally, and very possibly more interesting when creating a book: Internal links work as well. If an object in the book has been given an ID (which is now also more widely possible), you can link to it by prepending it with a # and using that as your link target (which is similar to how you might link to a named anchor in a simple html document). In its simplest form, this allows you to jump from page to page (which also is what the Jump element was originally designed to do), but with this system, much more power is added, by being able to link to not only pages, but also individual frames, and the reference and binary objects.

Linking to a page or a frame will make Peruse navigate to that location in the book, as you would expect (the way Jumps previously would link directly to a page). The idea there is to allow you to create branching narratives and other interactions of that type, like those you would find in a Choose Your Own Adventure book. Of course, ACBF is not a scripting system, and Peruse doesn't have one either; this is "just" markup, so it's not like you can create a full blown game here, in the modern sense, but even then, engaging narratives do not require score keeping for their interest :D

Say you've got a lovely CC-BY banner, and want to show the license? Pop the text in a Reference, and Jump away!

Linking to a reference or a binary object will make Peruse display that object in a popup or a page. An author can use this to create a glossary of terms, for example, or character reference art, a location map, or a world encyclopedia built right into the book itself. As someone who does writing and world building in their spare time, this in itself is something that i'm hoping to expand on greatly (how does having a pile of world-building focused templates for references sound to you? Sounds kind of neat to me, and i'd really like to see that happen :) ).

Something else Peruse does is keep track of all these internal links. Or, rather, the libacbf code does - we built that library as a non-gui Qt style library to be consumable without being bound to Peruse, but more on that at some later date (Tier 1 ACBF KDE Framework anybody? ;) ). What that means is that if you are at a location in a book which is linked to by something else, Peruse knows about it.

Right now, that information isn't used, but we might imagine things like building two-way navigation in the book (say you click a link, and you want to go back to where you came from; well with this we can make sure that happens), or even building entire navigation maps for a book (technically this tracking system for the internal links constitutes a directional graph, where each node is a location in the book, and the edges are the links). How such a thing would be presented visually, i'm frankly unsure, but as a basic idea it feels like it could be fun!

If this sounds like something you'd like to work with, get in touch - either here, or anywhere else you find me - i'm leinir basically everywhere, and we've got our own little corner over on Matrix as well. In short, get in touch, we'd love to hear from you! :)

---

The word of the day is: Emergent. Because that's the kind of behaviour these bits of functionality promises