Brad Templeton Home
The TVWish Program
Hard disk Video Recorders are great. The open source MythTV system allows you to change and improve the system, which is why I made this program that lets you record great movies whenever they come on, and a whole lot more. It's a new component of the PVR Revolution, this time about what you watch as well as how you watch.
What are the key things TVWish lets you do?
Many video recorder systems have a wishlist or search feature, which lets you program in a request for any show that matches a search string, even if it isn't showing in the 2 upcoming weeks you have listing data for. Thus you can enter a search for a movie, like "Tron" and your box will record it whenever it comes on, even months or years in the future.
Your TV schedule is full of great movies, including many you haven't seen. TVWish lets you create a "wishlist" from a wide variety of sources for movies, TV episodes and other shows, even if they are not yet present in your 2 weeks of advanced listings.
Run regularly, it will search for any of your wishlist items and queue them up to be recorded. You can build a large wishlist with hundreds of movies and shows, and even if they don't show up until next year, it will arrange for them to be recorded.
TVWish runs in the background, and can handle a very large list with more complex requests, plus live requests from outside.
Your schedule is also full of series that have many good episodes, but also have too many "worst episode ever" candidates to make them worth watching. TVWish lets you import a good parts only list of episodes for a series so you see only them. Fan sites are full of these lists. (For full details see the section on Abridging a TV Series.
In addition, TVWish contains a bold, perhaps revolutionary new feature. You can, in your wishlist, include another wishlist from out on the net using a web URL. This allows other people you trust, whom I call "critics" to create lists of good programs to watch, and your Mythbox will automatically record them. You can subscribe to as many critics as you like, and critics can include the results of other critics.
You can also store lists on your own machine and include them with a more traditional file include.
Today, professional TV critics at newspapers get advanced copies of TV shows on tape. They watch them, and put reviews in the newspaper before the show airs for ordinary viewers. Imagine such a critic you trust putting a show in their list if it's really good.
Of course, no professional critic provides this list yet, but already I am talking with one newspaper critic interested in doing this. Such a critic could be very powerful. Unlike the ordinary newspaper critic (who can write a review that if good, the studios hope you will read,) this critic might say, "If I love a show, 10,000 people will automatically record it and probably watch it."
Here are some possible uses for imported critic's lists:
All wishlist recommendations, and particularly critic's ones, are set at a lower priority than shows you request. So they won't ever stop your MythTV from recording a show you asked for. You can change the priority (and change it per-critic) if you like.
One main purpose of the system for me is movie wishlists. I have prepared a list of 100 or so top movies that I haven't seen. Almost all show up on TV sooner or later.
To start, you want to build your list of movies in an ordinary text file. Several are bundled with the program including these:
You can also visit web sites to help you build your own lists.
IDMB also has top databases for various genres, for box office and many other categories. Your favourite film critics probably have similar lists.
Just go to the lists you like, and use your mouse to select, copy and paste the list into your text editor. TVWish removes any ranking numbers and parses out the year of release to make sure you don't get remakes. (If you want remakes, remove the year or put in multiple entries.) As such, you don't have to clean them up the text you copied and pasted much, if at all. You can also copy and paste most tables into a spreadsheet like Gnumeric or Excel, and then select just the data you want and save it out as a text file.
Your goal is a text file, with one movie per line, with an optional year in parentheses, and if need be numbers before and after the name.
Next, go through the list and turn it into your wishlist. In other words, remove the movies you have already seen or know you don't want to see.
TVWish expects and exact match on the name, but case doesn't matter. If you don't know the exact match, you can use wildcard characters. The character "%" matches any string, and underbar "_" matches a single character. Thus if you want to match all Star Wars movies you could use "Star Wars%" or even "%Star Wars%" to match any movie with Star Wars in the title.
Quick Summary of major syntax
Lines in TVWish files are generally either wishes (movie, show or episode requests) or commands.
Each line in the file requests a program or programs to be recorded. They can begin with command word, colon and argument, or if this is not present a default command is executed on the line, usually "Movie." Optional keyword=value tags can be added to many command lines.
You may need to fix up cut-and-pasted lists a bit. Some of the titles may not be in the form your TV listings will use.
Sources like IMDB like to identify movies by the title in the language of the country of origin. That's fair, though they sometimes overdo it, for example listing "The Good, the Bad and the Ugly" -- which was shot in English -- as "Buono, il brutto, il cattivo, Il." You will need to put in a name more likely to appear in your own TV listings.
If using the default command, watch out for titles that start with a word and a colon! Quotes can fix this issue.
Download and run
The program is a perl script any user can run, but more commonly it's called from the cron as set up in the install docs. You can either pass it files of wishlists, or you can use a master file that includes other wishlists.
When you run it stand-alone, the typical usage is
TVWish does not create "Search" entries for you in Myth. You will only see entries for shows that show up in your listing data. You can delete those, or mark them as "Never Record" if you won't or can't remove them from the wishlists or critic's lists you are using.
Full Commands & Syntax
General lines in a TVWish file consist of a command, followed by a colon, and then a string argument. However, additional parameters and options can also be added before or after the argument. They are of the form Keyword=Value and the value must be in quotes if it contains spaces.
In addition, there is a default command if the line does not start with "Word:" or similar. That starts out as "Movie" which means that a title on a line by itself is a request to record a movie by that name.
However, you can change the default command and set some other parameters as you go.
Backslash (\) is the escape character. In the rare event you need a literal equals sign with characters on either side, you can escape it as in War\=Peace. You get \ with \\, but you don't need to double your backslash in front of "%" or "_" in titles.
Commands are either "wishes" that request shows for record or control commands that set variables, include files etc.
See the Suggestions section for description of the suggestions system and its commands.
As noted, you don't have to say "Movie:" -- this is normally the default command so a title on a line by itself works as a movie wish. In addition, the "-n" option will strip any numbers from the front and back of the movie.
If a movie title request ends with a year in parentheses -- in other words something like "(1985)" -- it is taken as the year of release of the movie. The request will match only that movie in that year, no remakes.
You can also add a parameter to specify the year more formally. Thus:
is the same as:
This wish is more generic, and will match any title, including movies or series, but primarily it is used for specials which don't fit under the Movie and episode searches. This is not used much because special titles can be hard to predict. Show: %Annual Academy Awards might pick up the Oscars every year but won't if they change the way they title the special. Sometimes it's better to just do these from the ordinary listings.
This wish tries to look for the first episode of the named series. It uses the "episode number." There is no set of rules for episode numbers, so it looks for numbers like 0, 1, 100, 101, 001, 1001, 01001 and so on.
Note that this means that for some series (the ones that use plain counting -- 1, 2, 3, 4 -- instead of 101, 102 ... 201, 202) you may also see episodes one-hundred and 101 with this request. A special test on matches of episode 100 and 101 checks to see if any lower epsiode numbers are present and does not record in this case. For shows that start their counting with 0, you will catch the first two episodes, which is often fine anyway.
The sure way to get a first episode is to know how that series numbers its episodes and use a manual episode number, or a manual subtitle if you know the title. The -e option to tvwish will help you do that. You can often find this on fan episode guides. Almost every TV show has an episode guide, just do a web search for that.
This will record the first episode of any series which is in the given category. Thus First: category="Science Fiction" will record the first episode of any SF show, including new ones, as long as they use one of the common episode numbering patterns. The Category names are the same ones Myth uses it its search.
Set: [environment variables]
While processing requests, there is an "environment" of settings like the default Series, Command or scores. You can set environment variables and they last until the end of the current file or critic page. You can also set them when including a file or critic page just for that set of wishes. You can set these variables with tags on the Set command, as well as all the other specialty commands dedicated to a particular variable.
As you'll see below, all these variables can also be set when including a file or Critic's list.
Episode: Episode-Title Series=Series-name
This wish will record a specific episode of a given series. Episode titles can usually be found on web fan sites with episode guides. You must bet the title exactly (though upper case and lower case don't matter,) or use the "%" wildcard as described in the movies section.
Will record the first episode of Buffy (which uses an unusual episode numbering system.)
If you have set a default series, then you can leave off the Series parameter. You can also set the default command (see below). In fact, that's the norm for a file devoted to episodes of a single series.
Series: Buffy the vampire slayer Episode: Welcome to the Hellmouth Episode: Once more with feeling
Episode: num=Episode-number Series=Series-name
This will record a specific episode number of a given series. This can be a way to get the first episode, or a specific one you see in an episode guide on the web. But you want to be sure of the numbering system in your listing data. Thus try to use perl tvwish -e series name as documented above to get a list of all current episodes. You should be able to work out the episode number pattern.
You can include a parameter indicating a rank for an episode. For example the "best episode ever" would have rank=1 on it. If you have set a minimum rank with the Series: command or other environment variable settings, the show will only be recorded if it is numerically lower (meaning ranked better than) that ranking number. This lets you use a complete file of all episodes of a show, but effectively ignore those worse than a certain ranking.
You can also provide a parameter with a score. These usually come from people who have voted on shows to give them numeric points or stars. You can set a minimum score to effectively ignore any episode scoring below that. Like the rank, it lets you import a complete file and filter it. Actually, almost all commands and wishes in the system can have a score tag on them and will be skipped if the score is less than the minimum.
As an alternative to an episode name or number, you can request all episodes that originally aired before the specified date. Just about any date format you can think of can be used, though you might want to put them in quotes.
Would request every episode of the Simpsons aired before December 1, 1995. You might use this if you started watching a show after it had been on for a while, and wanted to catch the early ones. However, doing an abridged series might make more sense. The magic token "%" is the wildcard character which matches any episode title.
This is just like "Episode" but if there is a ranking at the start of the episode line, or a numeric score at the end, they will be processed and removed. This allows you to more easily import lists with such ratings on them. This is for easy import, for more formal specification, tags are recommended. Ranks must be numeric but may be followed by English ranking suffixes like "1st."
Thus the following would work:
RatedEpisode: 23rd The mediocre one Series="Fine Series" RatedEpisode: The mediocre one 7.5 Series="Fine Series"
As being equivalent to rank=23rd or score=7.5.
You will almost surely be using this with "Set: Command=RatedEpisode" and Series so you can drop in a list of rated episodes into your text file as-is.
This takes the same flags as the episode test, but defines an episode title you wish to exclude. Ie. any requests to record it, even by pattern match, will be disregarded if they are later in the file. A typical use is to set up to record all episodes of a series except a given excluded set, by adding several "Exclude" lines and then one request for Episode: % which does a wildcard that matches all episodes.
Note that the rank and score tags can be used here, but their sense is inverted. That is to say, the Exclude will be ignored if it is above the minimum score, which gives the result that we want -- that the episode will be recorded if the score is above the minimum.
This is just like the "Set" command but it takes an untagged argument that is the series name, and is placed in the Series variable. A common usage form is:
Which sets the default command to be an Episode wish, and sets a default series for those wishes. You can follow this with a simple list of episode names.
This powerful environment variable lets you set a number to specify not to record any further shows in a given file if you have "missed" this many prior episodes in a row.
For example, some shows are in syndication at 2 or more different places in their life-cycle. Some mix repeats and new shows. You want to watch them in order. If your list is in order, then for every episode it tries to record, it checks to see if you have ever recorded it before (or if it's in your pending recordings.) If you have not, you don't want to record later episodes until you get this one, or you will be watching out of order.
Ideally you would say, "Don't record later shows if there are any I have missed." But due to conflicts this probably isn't possible. You may want to let it have a few missing episodes.
Another great use of this flag is for a series in syndication. Perhaps they show it every day, and the first episode comes around 3 times a year. You want to record your selections of it, but you want to wait until you get the first episode before starting. Adding this parameter will make that happen.
This will request a program using the program-IDs used by Zap2it Data Direct. This would primarily be used by Critics who want to be exact in specifying an upcoming show.
At least right now records the first episode of The Simpsons
SQL: partial SQL query
Here you can, if you are an advanced user, write wish with an SQL query in MythTV's program table. You will be writing only the WHERE part, and some clauses are already present and ANDed with your clause. Some examples:
SQL: title = "Doctor Who" AND subtitle LIKE "The Daleks" SQL: category_type = "movie" AND category LIKE "science fiction" AND stars > 0.9
For security reasons, this wish is denied to critics. I am curious about whether MySQL security is good enough that this could change.
Include: filename [environment settings]
Inserts another file on your system into the wishlist. Handy for building lists from different sources.
Note that "Include" can't be used in a critic's list.
You can provide tags to set all the environment variables found in "Set" above. These tags affect the environment only for the included file and its own inclusions. By setting things like the command and series, you can turn an ordinary text file with TV episode names into a TVWish wishlist.
Here we include a file of JAG episode names, possibly with ranks, and record only those in the top 20%. These recordings are 1 point lower than other TVWish requests.
Critic: web-URL [environment settings]
Perhaps the most revolutionary part of the system. Lets you import a set of wishlist requests from a text file located at that URL.
Would import my list of a few top TV episodes for you. Note I am not maintaining this as a service right now, it's just an example.
Only use critics you trust. The system does have safeguards against a malicious imported file (for example, critics can't include files on your system, but they can include other critics) but they are hardly well tested at present. Plus, you are letting these folks get control over your MythTV box, they can make it record shows, though at a lower priority setting than you. That can still take up your disk space and, at present, knock out other shows set to autoexpire.
Just as with Include: above, you can temporarily set environment variables for the Critic's list and all other lists included from it.
This command, with no arguments, stops scanning of the current file. This is mainly a way in a large file (one with all the episodes of a series) to tell it to not scan the latter part of the file because you are working on recording the earlier years for now.
Here are some tags that can be used on most wishes. Some are just comments but may have value in the future
The program depends on the unique "ProgramID" assigned each show by the DataDirect Service. Myth 0.17 and later generate a synthetic ProgramID if this is missing (ie. you don't get your data from DataDirect North America) which should work. Also, it depends on other DataDirect attributes, such as episode numbers and the fact that movies come with a "categorytype" of "movie" -- that's not true with some DVB feeds. Patches to work better with other listings are welcome.
Right now the program uses two different kinds of Myth recording requests. For non-series shows, like movies, it uses a "FindOne" recording, which will find any showing of the movie and adjust for conflicts. For series episodes, however, it uses a power search for the specific episode combined with a FindOne. There is a bug in 0.16 Myth whereby two or more power searches for different episodes of the same series make it display the later ones as not to be recorded. However, once the first one is recorded, the next becomes active and works.
Note you will want to go into the Setup for TV/Priorities and check the box that says that higher priority shows can be rescheduled to avoid conflicts. This means if one of your shows can be recorded at a later time to allow one of your wishes to be recorded, it will, but if they conflict, it won't.
As noted, in most cases an exact match is expected on Movie names, Series names and Episode names. However, you don't have to get the capitalization right. If you are worried, use the "%" wildcard. Of course, if you have seen listing entries, you know what to put in exactly.
Also as noted, the "First Episode" test is not exact. Possibly we will eventually build a database of the episode numbering styles of all major TV series, and be able to prepare first episode record requests.
Allow critic files to be in HTML so they are easier for them to build. Parse out the HTML paragraphs or table entries.
Allow files (and critic's files) to put in comments on why a show is recorded, and make this comment visible to the user when they look at the recorded show or queued recording.
Provide web interface and use mysql database, pre-loaded with sample lists. Let user check off ones they want to see/avoid, importing other lists. Let them add entries in web interface.
Cache critic's lists, and fetch them with an If-Modified-Since header to reduce load on critic's web sites that need fast update. (underway)
Consider supporting an XML format like RSS for critic's lists, so people can blog 'em.
Deal with non-DataDirect sources better
Abridged series subscriptions
A number of the commands and options above exist to allow you to watch an abridged "good parts version" of a TV-series that's in reruns. Life is too short to watch bad TV. I started doing this a few years ago manually and love it. Get reports from fan votes and critics on what the good (or bad) episodes of a series are, and watch only the good parts. Read web episode guides to learn any relevant plot details from turkey episodes.
You can read the details in this section on Abridging a TV Series.
If you keep your "mysql.txt" file other than the standard place /usr/share/mythtv/mysql.txt you will have to edit a line at the start of the program to show where it is. Otherwise the program should run fine on any system that runs mythfrontend or mythbackend. It only talks to the database, not to Myth itself.
If you combine from several lists, you might enter some duplicates. TVWish will handle this, but you may also wish to use the linux command "sort -u" which will sort your list and remove duplicate lines.
TVWish was briefly named MythWish and WishTV.
The lists, however, in pure form are very simple and easy to make:
Sample TVWish File
# Movies Midnight Cowboy (1969) Blade Runner (1982) Blood simple (1984) # Some first episodes First: The Prisoner First: The X-Files First: WKRP in Cincinnati # The first episode of any Reality show First: (Reality) # A specific request for a top episode Episode: Series="Buffy the vampire slayer" Becoming # Now include my big movie list Include: /home/me/etc/bigmovielist.txt # Now import the recommendations of a critic I like Critic: http://www.moviecritic.com/mythtv/watchthis.txt Priority=-1 Suggest-Learn: Suggest-Go: hours=4
For comments and bug-reports or patches, E-mail to the tvwish address at the templetons.com domain.