Day 8: More feed than I can eat in one sitting

Where to cast the pods

The first question is where the podcasts should be stored once downloaded. Unfortunately there isn’t a standard “podcasts” directory provided by any operating system like there is for Documents or Music. Even if there was, it would be quite presumptuous of any software to just immediately take over that directory’s management.

The simple but difficult answer is, I think, “it depends; you need to ask”. There isn’t going to be any one directory that makes sense for the majority of people. What makes this difficult is the lack of any sort of universal file-picker in the terminal. If this were a GUI application, I would have a standard widget available for a file picker, which would pop up a familiar UI to let someone select a directory. In the terminal, the most natural interface would be to ask the person using the software to type in a file path by hand. This wouldn’t be a very friendly interface. What if they make a typo early in the path and accidentally create a whole tree of directories they don’t want? Would they try to escape spaces etc. in the input? Should they? Etc.

I think the friendliest interface a terminal app could provide here would be some sort of fuzzy-search based file browser, similar to fzf (or fzy, which I prefer). On that note, here’s a shell config that any fellow fish shell users may find helpful:

function ctrlf
  commandline --current-token --replace -- (fd -t f | fzy)
  commandline --function repaint
end
bind \cf ctrlf

Drop this in your config.fish and it will let you type Ctrl + f in the middle of any shell command to execute a fuzzy file search in the current directory. Whatever you select will be entered at the cursor as if you had had typed it out. You could also replace fd -t f with find . -type f and fzy with fzf, if you prefer. I recommend fd though, since it will respect .gitignore.

Regardless of what the out-of-box experience should be for setting up your download location, a single download location wouldn’t be enough for my intended design. As mentioned in day 4, I want to be able to configure multiple download locations. This would let me synchronize my podcasts both to my laptop and to the SD card for my Tangara. I would also like people to be able to configure different download locations for different computers, using the same per-host conditional config mentioned in the same post.

For now, due to the previously mentioned constraints, I decided to use a “Podcasts” directory in the audio_dir() provided by directories.

let Some(user_dirs) = UserDirs::new() else {
    bail!("can't figure out where to store podcasts")
};

let Some(audio_dir) = user_dirs.audio_dir() else {
    bail!("can't find audio dir");
};

let podcast_dir = audio_dir.join("Podcasts");

Though in reality I didn’t actually implement the file downloading itself yet, so this isn’t yet used.

Starting up the reactor

Downloading the podcasts will, naturally, require accessing the network. Networks are, as a rule, slow, at least in terms of what you can do on the local machine. When interacting with them, you usually want some sort of strategy for doing other work in parallel, or at least doing multiple types of network I/O at the same time.

In Rust, the predominant strategy here is to use the reactor pattern, and the most popular reactor is the one provided by Tokio. Other reactors are available (I personally find smol quite elegant), but Tokio is the most popular, and allows me to use reqwest and http_cache_reqwest, which will simplify the HTTP logic in the short term.

Eventually I may instead swap to using hyper directly, which has far fewer transitive dependencies than reqwest (which is itself ultimately a wrapper around hyper), but this requires quite a bit more ceremony to set up, which I am trying to avoid before I have a functioning project. A mantra that has helped me with this (as well as to avoid other types of premature optimization) is that if you do the quick and slow thing first, then optimize, you get the satisfaciton both of quicker results and later of cleaning up.

Here we set up a reactor and spawn a set of tasks on it that will run in parallel. The tasks will each synchronize a single podcast feed as read from the config.

let rt = tokio::runtime::Builder::new_multi_thread()
   .enable_time()
   .enable_io()
   .build()
   .into_diagnostic()?;

let client = ClientBuilder::new(Client::builder().redirect(Policy::limited(5)).build().into_diagnostic()?)
   .with(Cache(HttpCache {
       mode: CacheMode::Default,
       manager: CACacheManager::new(http_cache_path, true),
       options: HttpCacheOptions {
           cache_options: Some(CacheOptions {
               shared: false,
               ..Default::default()
           }),
           ..Default::default()
       },
   }))
   .build();
let client = Arc::new(client);

rt.block_on(async {
   let mut feed_tasks = JoinSet::new();
   for feed in config.feeds() {
       let Ok(feed) = feed else {
           continue;
       };
       let feed = feed.clone();
       let client = client.clone();
       feed_tasks.spawn(sync_feed(client, feed));
   }

   feed_tasks.join_all().await;
});

Running out of time

Around here is where I ran out of time! So instead of continuing with my plan of downloading the first podcast episode from each configured feed, I will instead print out the URLs of each file in the terminal. At least this lets you download the files yourself if you like!

async fn sync_feed(client: Arc<ClientWithMiddleware>, feed: Feed) -> miette::Result<()> {
   let response = client.get(feed.url.clone()).send().await.into_diagnostic()?
       .text().await.into_diagnostic()?;
   let mut reader = Reader::from_str(&response);
   let mut buf = Vec::new();
   loop {
       match reader.read_event_into(&mut buf) {
           Ok(Event::Empty(tag)) => {
               match tag.name().0 {
                   b"enclosure" => {
                       let maybe_url = tag.attributes()
                           .find_map(|attr| {
                               attr.ok()
                                   .and_then(|attr| (attr.key.0 == b"url").then(|| attr.value))
                           });
                       if let Some(url) = maybe_url {
                           println!("{}", String::from_utf8_lossy(&url));
                       };
                   }
                   _ => ()
               };
           },
           Ok(Event::Eof) => return Ok(()),
           Err(err) => bail!("error parsing feed: {err}"),
           _ => (),
       };

       buf.clear();
   }
}

And now, having written that, my hands are starting to cramp up, so I think I’ll have to leave it there for today.