Using RSS and Newsboat (or another reader if you want)

I sometimes take a tour around the office where I work, and I also sometimes am able to see what people are doing on their phones when I'm out in the wild because people aren't even trying to hide what they're doing. They play videos at full volume in crowded spaces, they have the phones up to their faces when they're standing in the checkout line at the grocery store, and so on. I'm not trying to spy on anyone, but with phone screens getting bigger, it's sometimes harder to not see what they're doing than it is to avoid it. And what I see is that a lot of people seem to do very little other than open up facebook, or tiktok, or whatever, and just scroll and scroll and scroll and scroll, looking at stuff.

I'm not saying this is bad (that's a lie, I'm totally saying that this is bad). All of these sites have so many entities (let's be realistic here, it's not just people, it's businesses, it's content farms, it's AI robots, and so on) posting so many things that it would be impossible for anyone to look at it all. Instead a computer program will pick what it thinks you want to see. How does it do this? Ostensibly by keeping track of what you've engaged with in the past, and then trying to find other similar things across its vast trove of content that you're statistically likely to interact with. There's probably more to it than that, but the inner workings of these content-selection computer programs are closely guarded secrets, and you will never be able to know if the thing that showed up in your feed was something that was chosen because it was similar to a thing you looked at, or because someone behind the scenes decided to adjust one of the knobs to weigh something more heavily.

What if there was a way to only get the things you were interested in. What if you didn't need a site like facebook to do it for you? That technology has existed since the 1999 and is called 'RSS' or 'Really Simple Syndication'. You don't really need to know the nuts and bots of RSS to use it, though (unless you want to create your own RSS feeds). But what you will need is a news reader, and there are a lot to choose from. I'm going to be focusing on the terminal reader newsboat here, but feel free to give another reader a try. Other news readers include (but are not limited to) Liferea, The Old Reader, Microsoft Outlook, Mozilla Thunderbird, Elfeed for Emacs, and a ton more here that I don't have the time to list.

Installing newsboat

Newsboat is an application for Unix-like operating systems, that means you'll either need Linux, some kind of BSD, or the Windows Subsystem for Linux to install it. If you don't have one of those, you probably want to skip to the 'finding rss feeds to subscribe to' section, but you're going to be on your own putting them into your reader(s).

To install newsboat on your Unix-like system, use your system's package manager or app store to do it. Usually that's something like apt install newsboat on Debian-based systems, dnf install newsboat on RedHat-based systems, pkg add newsboat on some BSDs. You might need to consult your distribution's documentation if you've never installed packages before.

Once it's installed, go to your favorite terminal emulator and type newsboat and... nothing much will happen. It turns out that we need to do some configuation to get anywhere.

Configuring newsboat

You'll need to use your favorite text editor to customize Newsboat. On my system the file is in ~/.newsboat/config (the ~ is just a shortcut for the home directory of my current user, the full path is something like /home/username/.newsboat/config.

The Newsboat config file has a lot of options, most of which we won't be going over here. I'm going to go over my personal config, but you'll want to customize it to fit your preferences.

# Don't refresh feeds when newsboat starts
refresh-on-startup no
# automatically reload the configured feeds every 120 minutes
auto-reload yes
reload-time 120
# check 50 feeds for updates simultaneously 
reload-threads 50
# set the default browser for opening links to lynx
browser lynx
# download up to 4 files at a time when using podboat
max-downloads 4
# keep 14 days' worth of articles for each news feed
keep-articles-days 14
# open up media links with mpv and resize it so it fits on the screen
# this is good for when youtube creators upload in formats that are way too big
# then set the browser back to lynx afterward
macro v set browser "mpv --autofit-larger=25%x25%"; open-in-browser; set browser lynx

# open link in a private Firefox window, then set it back to
macro f set browser "firefox --private-window"; open-in-browser; set browser lynx

# reset browser to lynx
macro z set browser lynx
		    
# podboat config
# set the default media player for podcasts
player "mpv"
# delete played files when we're done with them
delete-played-files yes
# put the temp downloaded files in my home directory under Downloads/podboat
download-path ~/Downloads/podboat

Any line that starts with a # is a comment, so I put in some comments to explain what each line does. A few things to note:

• There's a companion program called podboat that you can use to listen to podcasts. We'll briefly touch on that below

• A macro is just a series of commands that you give to a computer to make it do a sequence of things. In newsboat you make a macro by typing macro and then the key you want to assign to the macro. Then you make a list of things you want the program to do, separated by a semicolon. For the 'v' macro above, we temporarily set the browser to mpv (which is a media player), then tell newsboat to open up the default link for the post using the default browser (which we just set to mpv), and then to set it back to lynx when we're done. To run a macro, you press the macro key (by default is the comma) and then the letter of the macro you want to run.

• lynx is a text-based web browser. It's blazing fast, and it doesn't support things like JavaScript or images. But it has a learning curve and it takes a while to get used to it, but you can get by with a few basic commands: press 'g' (for 'go') and enter a URL you want to visit, up and down arrows move between links you can interact with on the page, right arrow or enter follows the link, left arrow goes to the previous page you visited, 'q' quits. lynx usually does a good job of rendering the text of a page so you can usually figure out what's going on. I also have a macro for temporarily opening a link in a private firefox window if I really want to see the images in the article or otherwise need to use something that lynx doesn't provide, but it's usually not necessary if you're just interested in the text of the page anyway.

• I have the reload set for two hours, and with a decent number of feeds, this provides a big enough pool of articles that I have more articles than I'm able to read any time I take a break to check my reader.

  The best interval is something you're going to have to figure out on your own, but I'd caution about making the interval too low. I've seen configs that set the reload time to 1 minute, which is far too frequently. Setting your newsreader to poll every site you follow 1440 times a day to try to turn your newsreader into a worse twitter or facebook is the opposite of what we're trying do to. My suggestion is to add more feeds and to check them less frequently. You'll have plenty of stuff to go through.

Adding feeds to your newsreader

You might have seen this icon or something like it on websites. You might have even clicked on it and gotten a page full of something that looks like computer code.

This is the icon for rss feeds, or sometimes atom feeds, which are similar enough that we don't need to talk about the differences here. Odds are that your feed reader can understand both. And the 'computer code' you see is a specially-crafted information that tells your newsreader about recent updates to the website. Specifically what this information will tell to your news reader will depend on how the site operator configured their site. Sometimes you get the full content of each update, sometimes you get a teaser with a link to the full update, sometimes you get something else.

Sometimes the link to the feed won't be obvious, so you have to put on your detective hat and try to look for it. There are three main places to check:

1. Go to the address bar in your browser, and add /feed to the end of the address. i.e. if you are visiting https://example.com, try visiting https://example.com/feed If you get a bunch of computer code or your browser asks you to save a file, you're probably in the right place
2. Look at the site's source code. In most graphical browsers, you can press CTRL + U to look at the source code for the site you're looking at. It's way more than we have time to get into today, but you can search for the feed in this mess by pressing CTRL + F and typing in the first few characters you want. You might have to try a few different things before you find what you're looking for, like rss, xml, atom or sometimes rdf. It can take some digging, but some sites that don't advertise RSS support still offer feeds if you go looking for them. YouTube is a prime example. If you go to the main channel page of anyone on YouTube and view the page's source, you can find a link to a feed that you can put in your reader.
3. Sometimes websites will use third party services like Feedburner to provide rss services, and sometimes the link to those third party services will get broken or disrupted and the site owner seems disinclined to do anything about it.
4. Sometimes the website doesn't provide rss or any news feeds at all. There are some ways around this, but they're out of scope for this article

Finding Podcast feeds is sometimes tricky

Podcast newsfeeds are especially tricky because they tend to say that they're available "Wherever you get your podcasts" which means podcast aggregator websites, like Apple Podcasts or Spotify or Podbean or whoever, but not if you want to aggregate them yourself. If you can find the podcast on Apple Podcasts, for example, you can do the same CTRL + U thing and scour the source code for a link to the podcast. In Apple Podcasts it's near a piece of text that reads: "feedURL".

Now that you have the links to the feeds, what do you do with them?

Open up the urls file using your favorite text editor. It's probably located in the same directory as your config file, and is called, unsurprisingly, urls. Mine is located at ~/.newsboat/urls

The format is pretty simple. You put one feed per line, and any line that starts with a # is a comment. You can also add tags to feeds by putting a space after the address of the feed

~/.newsboat/urls
	
# this is a comment
https://wyrm.org/rss.xml wyrm.org howtointernet

Once you have your urls file full of all the urls you want to check, you're ready to start newsboat.

You'll see a screen that has a list of the feeds you're following, one on each line, with a number indicating how many new items are in each feed, and how many total items there are. Press the up and down arrows to select a feed, and the enter key to look at the list of articles. Pick and article with the arrow keys and then press enter again to read it. If you want to open the article in your browser, press the 'o' key to open the main link, or one of the number keys to open any of the other links that might be present (they're numbered for your convenience). If the feed you're checking out is a podcast, you can press the 'e' key to 'enqueue' it into Podboat for listening later.

You can also do fancy things like set up meta feeds that sort content according to criteria that you define, but that's getting a little too advanced for this tutorial, which is already teetering on the brink of being too long. Checking the documentation is also a good idea.

A brief mention about podboat

I mentioned that there is a podcast application included, and it's called podboat. In the newsboat interface you can go to a news item and, if it's a podcast episode, press 'e' on your keyboard to enqueue it into podboat. Then you either close newsboat or open a new window and run podboat. podboat has a very similar interface to newsboat. You use the arrows to navigate to the podcast you want to listen to, press 'd' (that's a small 'd') to download the episode, 'p to play it, and 'D' (that's a capital 'D') to delete the episode when you're done with it.

Congratulations! You're taking charge of your online experience

It may seem daunting and even a little bit overwhelming at first to be in charge of your own experience online. And I'll admit that it's a lot more work to curate your own experience. But the result is you deciding what you want to see, read, and listen to, and not some computer program running somewhere.