Solved how to get free guide data into WMC

Global guide related discussion. All Windows versions.
Post Reply
nhtvfan

Posts: 2
Joined: Mon Jan 06, 2020 12:15 am
Location:

HTPC Specs: Show details

Solved how to get free guide data into WMC

#1

Post by nhtvfan » Mon Jan 06, 2020 3:59 pm

If you're like me, and you want to continue to use WMC, but you don't want to pay for guide data then here's what I did to get Guide Data for free into WMC.
I was surprised to learn that there wasn't one complete set of instructions to describe and explain how to get free TV listings. So for the purpose of making it easier for anyone to learn how to get and use free tv listings, below are the steps.

I hope someone will amend this thread with a simpler solution. But here's a start.

Note that when you see below "command used", it implies that you open a windows command prompt(windows key + R, then type "cmd", click OK)
Also note for beginners, when you see the black command prompt, don't forget to use the change directory command.
You'll need to change from the current location to the program's folder by using a command like "cd C:\Users\Public\zap2xml"

This a summary of the software(freeware) needed:
--------------------------------------------------------------
EPG123 v1.3.4.0
instructions: epg123.garyan2.net/install
program: epg123.garyan2.net/download
zap2xml v2018-12-01
instructions: zap2xml.awardspace.info
program: zap2xml.awardspace.info
xmltv-to-mxf-0.0.2.jar
instructions: github.com/raydouglass/xmltv-to-mxf
program: github.com/raydouglass/xmltv-to-mxf/releases/tag/v0.0.2
Java v8 update231(required to run *.jar, check your system first, if installed do nothing, if not, install java runtime)
program: java.com/en/download/
loadmxf (WMC Windows 6.1.7600.16385)
instructions: loadmxf /?
program: C:\windows\ehome\loadmxf.exe (installed on all Windows with WMC)


Process Summary
---------------------
I used EPG123 to perform the equivalent of a fresh re-install of WMC. It reinitializes, erases, & cleans up WMC databases.
I used EPG123 to perform a fresh TV setup in WMC.
But note that this can be performed without this program. However, EPG123's instructions describe what to do.
I used EPG123 to manually create a channel map.
But note that this too can be performed without this program.
You can use WMC to manually recreate the channel map.
Instructions on how to do this are described in detail by github.com/raydouglass/xmltv-to-mxf
I used to zap2xml to fetch free tvlistings from zap2it.com to create a file: xmltv.xml.
This requires a free registration. No annual fees.
Alternatively, you could use tvguide.com as your free source.
Also note that more details can be retrieved, such as original air date by using the "details" option.
I used a java program "xmltv-to-mxf-0.0.2.jar" to convert xmltv.xml -> WMC.mxf
I used EPG123 to manually import the tv listings file: WMC.mxf
Or, if this fails you can use MSFT's WMC MXF loader: C:\windows\ehome\loadmxf.exe

Detailed Explanation
--------------------------
EPG123 was downloaded and installed.
Then I opened "EPG123 Client Tool" and followed the EPG123's website instructions to reset WMC.
I used the "Client Setup" button which displayed a 3 button prompt.
The 3 step process was followed to properly re-initialize WMC.

Note that when using Button 2, WMC is started and I proceeded through the TV setup process as instructed.
Once the UHF channels were scanned and saved, I noticed some channels were missing.
So I had to stay with in WMC's setting to manually add some channels.
I had to add VHF channels, which are not detected by WMC's TV Setup.

However, when performing EPG123's step 3, an error message appeared.
EPG123 tried to find a *.mxf, but since none exists.
So I moved on to learn how to create this missing file. But first I needed a file containing tv listings.

Now it was time to create a tv listings file, and I found & used this program (zap2xml.exe) to accomplish this task.
I downloaded and installed this program to this DIR: C:\Users\Public\zap2xml
This program allows you to get tv listings from 2 websites that provide tvlistings for free.
I elected to use zap2it.com as my web source, not tvguide.com

Following the instructions for zap2xml ...
I registered as a user on zap2it.com
logged in and setup my channel listings.
used zap2xml to fetch listings and create an XMLTV formatted file: xmltv.xml

command used: C:\Users\Public\zap2xml\zap2xml.exe -U -u EMAIL@address -p password -o C:\Users\Public\zap2xml\xmltv.xml

Now I needed to convert the xmltv formatted TV listings to WMC's native format *.mxf
I found and downloaded this program: xmltv-to-mxf-0.0.2.jar
This program converts the xmltv formatted TV listings to WMC's format *.mxf

But since this a java program, I also needed to download and install java.
Once java was installed, I used "xmltv-to-mxf-0.0.2.jar" to convert the xmltv file, but this process failed.
It displayed the following error: "Error in MXF: Invalid byte 2 of 3-byte UTF-8 sequence."
I found the solution by adding an option: -Dfile.encoding=UTF-8
command used: "C:\Program Files\Java\jre1.8.0_231\bin\java" -Dfile.encoding=UTF-8 -jar C:\users\public\zap2xml\xmltv-to-mxf-0.0.2.jar C:\users\public\zap2xml\xmltv.xml -o C:\users\public\zap2xml\WMC.mxf

At this point, I now have a file that WMC can use: C:\users\public\zap2xml\WMC.mxf

Now I re-opened the "EPG123 client tool" and clicked the "manual import" button.
This time the file imported without any error messages.(b/c I had an *.mxf to import)

Here's a detail worth noting. WMC can't be actively recording when EPG123 updates the Tv listings.

At this point, I restarted WMC and WMC's TV guide now displayed program names.
However, I noticed a minor problem. I noticed that some metadata was missing, such as the original air date.

Fortunately, there was an easy solution to this problem.
All I had to do is add another option to include program details to the zap2xml command.
command used: zap2xml -D -U -u EMAIL@address -p password -o "C:\Users\Public\zap2xml\xmltv.xml"

But of course this meant that the subsequent steps needed to be repeated.
This meant that I had to regenerate the *.mxf file using this command again:
command used: "C:\Program Files\Java\jre1.8.0_231\bin\java" -Dfile.encoding=UTF-8 -jar xmltv-to-mxf-0.0.2.jar C:\users\public\zap2xml\xmltv.xml -o C:\users\public\zap2xml\WMC.mxf

And I also had to re-import the file into WMC.
But when I tried to use the "EPG123 client tool" and used the "manual import" button, this time the import failed.
Fortunately, there was another easy solution. This time the solution was to switch to WMC's import tool:
command used: C:\Windows\ehome\loadmxf -i C:\users\public\zap2xml\WMC.mxf

Now I restarted WMC and WMC's TV guide displayed the program names, and contained more detailed metadata, such as the original air date.

As this point, the only thing to do was to automate these steps.
So I created a simple batch file b/c I'm using Win7x64 Home, but a VBS or a powershell script would also be an option.
Example batch file contents:
cd C:\Users\Public\zap2xml
:: note that zap2xml's default fetches 7 days
:: fetch 14 days of TV listings including details(ie:origAirDate). output file = xmltv*.xml
zap2xml -D -d 14 -U -u email.address -p password -o "C:\Users\Public\zap2xml\xmltv.xml"
:: convert xmltv.xml to *.mxf. output file = tv*.mxf
:: without option: -Dfile.encoding=UTF-8 command fails displays error msg = Error in MXF: Invalid byte 2 of 3-byte UTF-8 sequence.
"C:\Program Files\Java\jre1.8.0_231\bin\java" -Dfile.encoding=UTF-8 -jar xmltv-to-mxf-0.0.2.jar xmltv.xml -o tv.mxf
:: import new TV listings.
C:\Windows\ehome\loadmxf -i tv.mxf


Then it's simply a matter of creating a new task within "Task Scheduler" to execute this script routinely.
I set up a task to execute once a week to keep WMC's "TV Guide" TV listings updated automatically.

Hope this works for you. And if not, hopefully this helped you to learn the steps necessary to getting your tv listings updated.
Additionally, if you know of an easier method. Please share.

User avatar
Scallica

Posts: 2769
Joined: Mon Jun 06, 2011 7:09 pm
Location: USA!

HTPC Specs: Show details

#2

Post by Scallica » Mon Jan 06, 2020 5:04 pm

All of that to avoid $2.08/month (a slice of pizza or a cup of coffee) :wtf:
HTPC Enthusiast / Forum Moderator - TGB.tv Code of Conduct

testmonkey

Posts: 28
Joined: Sun Jan 20, 2013 1:45 pm
Location: Milwaukee Wisconsin 53220 Spectrum

HTPC Specs: Show details

#3

Post by testmonkey » Mon Jan 06, 2020 6:48 pm

Agreed with Scallica

Just the right-up???
Standing on the Shoulders of Giants

jonwz

Posts: 68
Joined: Sun Sep 18, 2016 3:03 pm
Location:

HTPC Specs: Show details

#4

Post by jonwz » Mon Jan 06, 2020 7:33 pm

Also agree with Scallica and Testmonkey.

I'm an old replaytv user and resisted Schedules Direct for years in favor of zap2xml. But SD's guide data is MUCH better. Now that I'm on 7MC with cablecard and comcast, I'm spending a gazillion hours trying to prune down the listing to the exact channels I receive from comcast (with no duplicates). I have no problem paying $25/year to SD and donating some coin to Gary for EPG123.

tonywagner

Posts: 176
Joined: Thu Feb 04, 2016 1:53 am
Location:

HTPC Specs: Show details

#5

Post by tonywagner » Mon Jan 06, 2020 11:52 pm

nhtvfan wrote:
Mon Jan 06, 2020 3:59 pm
I was surprised to learn that there wasn't one complete set of instructions to describe and explain how to get free TV listings.

...

I hope someone will amend this thread with a simpler solution. But here's a start.
I previously shared instructions for this exact process:

viewtopic.php?p=133930#p133930

tonywagner

Posts: 176
Joined: Thu Feb 04, 2016 1:53 am
Location:

HTPC Specs: Show details

#6

Post by tonywagner » Tue Jan 07, 2020 12:11 am

nhtvfan wrote:
Mon Jan 06, 2020 3:59 pm
However, I noticed a minor problem. I noticed that some metadata was missing, such as the original air date.

Fortunately, there was an easy solution to this problem.
All I had to do is add another option to include program details to the zap2xml command.
command used: zap2xml -D -U -u EMAIL@address -p password -o "C:\Users\Public\zap2xml\xmltv.xml"
First of all, note that you are possibly breaking the terms of service for Zap2It by downloading their data in this fashion.

Secondly, note that the "-D" program details command for zap2xml makes a separate request to the Zap2It server for every single program, which can easily number in the tens of thousands. This could be considered as an abuse of their service.

If you omit that program details option for zap2xml, you get pretty much everything the same except for the original air date, and the xml-to-mxf script is still smart enough to label all repeats with a past generic original air date (Jan 1, 1970), so WMC knows they aren't new.

At minimum, I'd recommend dropping that program details option in your usage to reduce the load on the Zap2It servers.

adam1991

Posts: 2877
Joined: Sat Jun 11, 2011 2:31 pm
Location:

HTPC Specs: Show details

#7

Post by adam1991 » Tue Jan 07, 2020 12:19 am

Scallica wrote:
Mon Jan 06, 2020 5:04 pm
All of that to avoid $2.08/month (a slice of pizza or a cup of coffee) :wtf:
Wait--he forgot to mention the time and effort he spends to register here under various different names.

Rabbitears

Posts: 3
Joined: Tue Jan 07, 2020 1:38 pm
Location:

HTPC Specs: Show details

#8

Post by Rabbitears » Tue Jan 07, 2020 1:52 pm

Maybe North American folks can save a whole $5/year cobbling something together with xmltvlistings.com
and mc2xml.awardspace.info

sgbroimp

Posts: 134
Joined: Mon Nov 21, 2016 5:49 pm
Location:

HTPC Specs: Show details

#9

Post by sgbroimp » Tue Jan 07, 2020 6:05 pm

Scallica wrote:
Mon Jan 06, 2020 5:04 pm
All of that to avoid $2.08/month (a slice of pizza or a cup of coffee) :wtf:
Yes. Looks like he has already done much more than $25 worth of work just writing all that. Penny wise.....etc.

User avatar
garyan2

Posts: 5530
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#10

Post by garyan2 » Tue Jan 07, 2020 7:09 pm

Rabbitears wrote:Maybe North American folks can save a whole $5/year cobbling something together with xmltvlistings.com
and mc2xml.awardspace.info
I hadn't heard of xmltvlistings. Looks promising, though still not free, which means it could be pretty good.

Everything with mc2xml of course is subscription based as well.
- Gary
Keeping WMC alive beyond January 2020. http://epg123.garyan2.net

kc8emd

Posts: 27
Joined: Wed Aug 15, 2012 1:11 pm
Location: Detroit

HTPC Specs: Show details

#11

Post by kc8emd » Fri Jan 10, 2020 2:31 pm

Thanks for this write-up. I have signed up for Schedules Direct for a year, but I am looking for a long term no cost guide solution and I think this is worth a try.

User avatar
DavidinCT

Posts: 1527
Joined: Mon Feb 13, 2012 3:45 pm
Location:

HTPC Specs: Show details

#12

Post by DavidinCT » Sun Sep 13, 2020 3:41 pm

tonywagner wrote:
Mon Jan 06, 2020 11:52 pm
nhtvfan wrote:
Mon Jan 06, 2020 3:59 pm
I was surprised to learn that there wasn't one complete set of instructions to describe and explain how to get free TV listings.

...

I hope someone will amend this thread with a simpler solution. But here's a start.
I previously shared instructions for this exact process:

viewtopic.php?p=133930#p133930
I've been using this process from the kill of the guide... I have it download the guide 3 days a week (at 2am), images seem fine, have not missed a single show. I have it all automated, I have not actually thought about in almost 3 months. It's working that good.

I DID mark channels in Zap2It as favorite, so it does not download EVERY channels information.

I only wanted to test it, and see how long it lasted, and still today, perfect guide data, channel logos work, images from shows are pretty much dead on.

Nice for free option...
-Dave
Twitter @TheCoolDave

Windows Media Center certified and WMC MVP 2010 - 2012

Green Lantern

Posts: 26
Joined: Thu Nov 07, 2019 7:45 pm
Location:

HTPC Specs: Show details

#13

Post by Green Lantern » Mon Sep 21, 2020 5:00 pm

DavidinCT wrote:
Sun Sep 13, 2020 3:41 pm
tonywagner wrote:
Mon Jan 06, 2020 11:52 pm
nhtvfan wrote:
Mon Jan 06, 2020 3:59 pm
I was surprised to learn that there wasn't one complete set of instructions to describe and explain how to get free TV listings.

...

I hope someone will amend this thread with a simpler solution. But here's a start.
I previously shared instructions for this exact process:

viewtopic.php?p=133930#p133930
I've been using this process from the kill of the guide... I have it download the guide 3 days a week (at 2am), images seem fine, have not missed a single show. I have it all automated, I have not actually thought about in almost 3 months. It's working that good.

I DID mark channels in Zap2It as favorite, so it does not download EVERY channels information.

I only wanted to test it, and see how long it lasted, and still today, perfect guide data, channel logos work, images from shows are pretty much dead on.

Nice for free option...
Since you've also been running this setup for quite awhile, have you encountered the following issue? An episode initially gets imported into the guide with a generic description (for example, indicates it will be an NFL game but doesn't say which teams yet) but even though an updated description later appears on the Zap2it website, subsequent update runs still leave the original description in the WMC guide? I've tried changing the -N flag to 14 (the same as the # of days of guide being fetched, figuring this would result in it not using any cached data) and I also deleted the cache folder. But even after running another update, the old description continues showing in my WMC guide. Any ideas how to resolve this?

User avatar
garyan2

Posts: 5530
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#14

Post by garyan2 » Mon Sep 21, 2020 5:29 pm

Green Lantern wrote:
Mon Sep 21, 2020 5:00 pm
Since you've also been running this setup for quite awhile, have you encountered the following issue? An episode initially gets imported into the guide with a generic description (for example, indicates it will be an NFL game but doesn't say which teams yet) but even though an updated description later appears on the Zap2it website, subsequent update runs still leave the original description in the WMC guide? I've tried changing the -N flag to 14 (the same as the # of days of guide being fetched, figuring this would result in it not using any cached data) and I also deleted the cache folder. But even after running another update, the old description continues showing in my WMC guide. Any ideas how to resolve this?
This is one of the "not-quite-right" quirks using this method of updating the guide. You may have seen a few other quirks. I can't recall whether this method has the risk of scrambling all the channel listings or not but I did do a check and provided feedback on it.

Speaking to this specific issue, this is due to logic WMC uses to determine whether a program will get updated or not regardless of what the imported MXF file says. An immediate option would be to install EPG123 (client only), run the client, and click the [Rebuild WMC Database] and select your MXF file to import the guide listings into a clean database. For the long term, if you don't want to do that over and over, you might think about downloading fewer days to increase the likelihood that the listings will be correct and won't be generic. You will still have this issue for the late night talk shows unless you reduce the number of days to 3 or something.

FYSA: The Gracenote/Zap2it lawyers apparently contacted the creator of zap2xml and evidently disapproved of his software. That is why it is no longer available online from his website... in fact the website isn't there anymore. It's just a matter of time before Zap2it changes their website and that tool will no longer work.
Last edited by garyan2 on Mon Sep 21, 2020 8:24 pm, edited 1 time in total.
- Gary
Keeping WMC alive beyond January 2020. http://epg123.garyan2.net

Green Lantern

Posts: 26
Joined: Thu Nov 07, 2019 7:45 pm
Location:

HTPC Specs: Show details

#15

Post by Green Lantern » Mon Sep 21, 2020 8:02 pm

garyan2 wrote:
Mon Sep 21, 2020 5:29 pm
Green Lantern wrote:
Mon Sep 21, 2020 5:00 pm
Since you've also been running this setup for quite awhile, have you encountered the following issue? An episode initially gets imported into the guide with a generic description (for example, indicates it will be an NFL game but doesn't say which teams yet) but even though an updated description later appears on the Zap2it website, subsequent update runs still leave the original description in the WMC guide? I've tried changing the -N flag to 14 (the same as the # of days of guide being fetched, figuring this would result in it not using any cached data) and I also deleted the cache folder. But even after running another update, the old description continues showing in my WMC guide. Any ideas how to resolve this?
This is one of the "not-quite-right" quirks using this method of updating the guide. You may have seen a few other quirks. I can't recall whether this method has the risk of scrambling all the channel listings or not but I did do a check and provided feedback on it.

Speaking to this specific issue, this is due to logic WMC uses to determine whether a program will get updated or not regardless of what the imported MXF file says. An immediate option would be to install EPG123 (client only), run the client, and click the and select your MXF file to import the guide listings into a clean database. For the long term, if you don't want to do that over and over, you might think about downloading fewer days to increase the likelihood that the listings will be correct and won't be generic. You will still have this issue for the late night talk shows unless you reduce the number of days to 3 or something.

FYSA: The Gracenote/Zap2it lawyers apparently contacted the creator of zap2xml and evidently disapproved of his software. That is why it is no longer available online from his website... in fact the website isn't there anymore. It's just a matter of time before Zap2it changes there website and that tool will no longer work.
Thanks for the info. Out of curiousity, is using EPG123 to [Rebuild WMC Database] the equivalent of deleting the mcepgg database (.db) file and restarting WMC? TIA

User avatar
garyan2

Posts: 5530
Joined: Fri Nov 27, 2015 7:23 pm
Location:

HTPC Specs: Show details

#16

Post by garyan2 » Mon Sep 21, 2020 8:22 pm

Green Lantern wrote:
Mon Sep 21, 2020 8:02 pm
Thanks for the info. Out of curiousity, is using EPG123 to [Rebuild WMC Database] the equivalent of deleting the mcepgg database (.db) file and restarting WMC? TIA
I would say yes, except you're not guaranteed that what is restored will be the most current using that method. Using EPG123 to rebuild, it will force a backup of your tuner setup, your recording requests, your lineup mappings, your favorites, and your channel logos. It will then delete the database, create a new one, and restore all the above. All that is needed is to import your guide listings and you're back in business. The whole process takes but a few seconds.

EDIT: I should mention that if you haven't done very recent changes to the listed items above that are backed up, then the backup may not be necessary. I just like to be thorough.
- Gary
Keeping WMC alive beyond January 2020. http://epg123.garyan2.net

tirebiter

Posts: 20
Joined: Mon Jul 03, 2017 8:33 pm
Location:

HTPC Specs: Show details

#17

Post by tirebiter » Tue Sep 22, 2020 5:13 pm

I agree with others. I am happy to pay the $25/yr pittance to SD. I hope they continue to offer this valuable service for many years to come.

They also offer a number of utilities and code samples for end users to customize their own use of the guide data. It's very useful (although sometimes challenging).

Post Reply