v1.7.0.2 Release

[garyan2]

This is the production release to include the proxy server service between WMC and Schedules Direct for image downloads. It also serves as an HTTP server for the MXF, XMLTV, and station logos as well as the trace.log and server.log files.

Though not really needed for stand-alone full mode setup, I would still recommend enabling the cache. There a quite a few benefits doing so with the only downside being taking up space on the drive. If there are any questions or issues, please let me know.

There is a slight performance hit in certain situations, but not anything that should impact you very much. When first building your WMC cache, whether using the EPG123 cache or not, if you open the movie guide WMC will try to download 24 images at once and the service can't respond quick enough to get them all. This is true for anything you are trying to do that may want to open tens of connections to the service for images. Just scrolling through the guide will certainly not be an issue.

[StinkyImp]

Though not really needed for stand-alone full mode setup, I would still recommend enabling the cache.

In my current "portable" install there's a folder in my ..\epg123\cache\ structure that has a file named "epg123cache.json". Is this the cache you're referring to?

If so, will this be the same cache v1.7+ will use?

If not, is there a mechanism to allow us to place the cache in a different location?

Thank you Gary!

[garyan2]

The image cache will be in ..\epg123\images\. The ..\epg123\cache\epg123cache.json is untouched by this new feature.

Currently no mechanism to change the location of the image cache.

[benhase]

Gary,
I am currently running epg123 version 1.6.7.0 on one Windows 10 PC.
Once a day I manually run epg123 to create the two output files: epg123.mxf and epg123.xmltv.
I use epg123.xmltv on the Windows 10 PC with NextPVR. WMC is not installed on the Windows 10 PC.

I use epg123.mxf on six different Windows 8.1 PCs with WMC installed, by manually running a simple .bat file.
The .bat file copies epg123.mxf to the local Windows 8.1 PC and then runs command C:\Windows\ehome\loadmxf -i epg123.mxf.
I don't run the Client on any PC. I don't run any scheduled tasks.

Will this process still work with epg123 V1.7.0.2?

I think you said Schedules Direct registers my IP address for 24 hours when I run epg123 on Windows 10 to create the Guide.
So when the six Windows 8.1 PCs running WMC request an image from SD (from my same IP address) SD will service the request.
I hoping I don't need the new epg123 proxy server service between WMC and Schedules Direct, image cache, tokens, etc.
I'm trying to keep my six Windows 8.1 PCs with WMC installed simple.

[garyan2]

The process will still work, but you won't be getting any images in WMC from Schedules Direct, nor your station logos, if the service is not running on your Win10 machine that creates the MXF and XMLTV files.

The timer for image access will start when you successfully login to SD to download your guide data. It will satisfy the access requirement for all your machines on your network. But if the service is running on your Win10 machine, then it will be the only machine to reach out to SD unless you install the server components on another machine as well.

I am not sure why you don't want to use any scheduled tasks. By not using the client on the Win8 machines, you will have problems in the future.

• you are not performing any database maintenance actions
  • garbage cleanup
  • backups
• you are not performing a verification of guide listings
• any lineup changes will have to be dealt with on all machines manually

I think you've gone to an extreme to keep your Win8 machines "simple". Having the client installed, which does not reside in memory nor have any hooks into WMC, is pretty simple. The client does what you are doing manually, plus it performs database maintenance for you. Not only simple, but beneficial.

As for the service, unfortunately this is the way it needs to be. In a future SD API update, the image requests will need to include a valid login token for SD in order to access them. The service will ensure that happens. It also makes sure that WMC will not download and cache the same images over & over again which not only takes drive space needlessly, but it will also count against a download limit SD has put in place. Once you hit the limit then you won't be able to download any images until that timer has expired. Any images downloaded on your NextPVR also counts against you, so you could potentially meet that limit pretty quickly depending on your usage between NextPVR and 6 WMC machines.

I most certainly would recommend running the service on the Win10 machine and enabling the image cache. Continuing to not use the client on your Win8 machines is totally up to you.

EDIT: I should also recommend that if you continue to manually run the .bat file to import guide listings on your clients, that your batch file look like the below:

Code: Select all

START /WAIT c:\windows\ehome\mcupdate.exe -b -dbgc -updateTrigger
c:\windows\ehome\loadmxf.exe -i epg123.mxf
c:\windows\ehome\ehPrivJob.exe /DoReindexSearchRoot
c:\windows\ehome\mcupdate.exe -PvrSchedule -nogc

The last 2 lines are to reindex your database so your searches work and your pvr schedule gets updated. I'm not sure if you are doing that already or not. The above is effectively what the client does without any automatic mapping of guide listings to your WMC tuner channels or verification.

[garyan2]

Sorry for the edits, but the example .bat file I posted above will have you performing garbage cleanup every time. You may want to separate that first line into a different bat file and run at least once a week. The client will do it every 5 days.

[SwissCheese]

Hi

I installed version 1.7.0.2 yesterday and have
had a day to play with it.

Overall the installation went very smoothly and
the new features look great!


Some comments....

1) I've been using the portable version for years and
was worried the installer might mess up my old-school
c:\epg123 folder because it's not in c:\Program Files.

It turned out there was no need to worry, the installer was
aware of my old epg123 folder and prompted to use it by default.
Nice!

2) I wasn't expecting the "Custom Install" options. They're
very useful, and yes I used them.

3) In Config, I deleted the old task and created
a new one just in case it was a necessary step.
It remembered my previous wake time, that's a nice touch.

4) The new image retention-time options are great!

5) After doing a save-and-execute, I waited a bit,
opened the guide in WMC, selected the TCM channel,
and scrolled down through its 14-day schedule.

As with my old epg123 version, guide images were slow to
populate because they were being downloaded via the internet.

Once all the images were downloaded, I closed/reopened WMC
and scrolled down through TCM again.

With the old version, images would be slow to appear again because
they were being re-downloaded, but with this new version images
loaded from the cache immediately with no perceptible delay
even when I scrolled down as fast as I could.

I also tried turning off the internet connection. The old version
wouldn't be able to re-download images. but with the new
version guide images are still there. Very nice!



Some questions....

1) When or how often are cached image files checked
for expiration and deletion?

I see a file in the /images folder that has
"last accessed" info for images, so presumably
the service checks periodically for expired files.

I'm just wondering if expired files are deleted at a specific
time of day on some specific event.


2) If I set the file properties of a cached image jpg to "readonly",
will epg123 panic when it tries to delete the image when it
eventually expires?


3) Are images from TMDB cached?

I'm guessing not, because among the many local image links in
my mxf are still a few dozen internet links to tmdb.org.

<GuideImage id="i3638" imageUrl="http://Higgs:9009/image/e7...59.jpg" />
<GuideImage id="i3639" imageUrl="http://Higgs:9009/image/4a...81.jpg" />
<GuideImage id="i3640" imageUrl="http://image.tmdb.org/t/p/w342/9J...hv.jpg" />


Thank you!

[garyan2]

1. The service will cleanup the cache when either the service stops, or when there is a new token detected. So at least once a day based on your scheduled update.
2. If it fails to remove the file, then it will also keep the cache entry in the json file but will try to delete every time #1 is met; otherwise it will be fine with it.
3. No tmdb files are cached. These images are strictly from Schedules Direct. You WMC cache (internet cache) will keep the tmdb images until your browser/OS removes them.

Whenever an image is requested by WMC for a refresh, that "last accessed" value is updated so the counter for determining when to remove it from cache starts over... be it 30 days or 30 years.

[SwissCheese]

OK.
Thank you again!

[garyan2]

Coming in the next version ... a Geek Status Page!

[benhase]

Gary,
Thank you for your reply #5 above.
So, what I have now (epg123 version 1.6.7.0 Server on one Windows 10 PC) will continue to work (with images) until SD makes the interface change that requires a token on each image request.
Just FYI, I do not use "station logos" today.

Once I install epg123 version 1.7.0.x on my one Windows 10 PC (where I build epg123.mxf and epg123.xmltv today):

1. All of the image links in the output files epg123.mxf and epg123.xmltv will change:
from: https://json.schedulesdirect.org/201412 ... abcdef.jpg
to: http://<HOSTNAME>:9009/image/0123456789abcdef.jpg where <HOSTNAME> is my Windows 10 PC.

2. epg123 version 1.7.0.x will also install "the service" on my one Windows 10 PC.
Where: "the service" receives image requests from my six Windows 8.1 machines (running WMC), adds the token, and forwards the image request to SD.
"the service" also implements an optional Image Cache to eliminate duplicate requests for the same image from SD.

3. I need to leave my Windows 10 PC running to service Image Requests from my six Windows 8.1 PCs (running WMC).
If it is not running, the guide will still display on the Windows 8.1 PCs, but no images will be available (unless Cached on the 8.1 PCs).

4. I can still use my simple .bat file on my six Windows 8.1 PCs to copy and load new guide files (via C:\Windows\ehome\loadmxf -i epg123.mxf)
The .bat file already has the four commands you recommended. I do not run the first line every time. However, I do NOT have the -b in the first line. What does that do?
START /WAIT c:\windows\ehome\mcupdate.exe -b -dbgc -updateTrigger
c:\windows\ehome\loadmxf.exe -i epg123.mxf
c:\windows\ehome\ehPrivJob.exe /DoReindexSearchRoot
c:\windows\ehome\mcupdate.exe -PvrSchedule -nogc

5. I will see a cool "Geek Status Page" on my Windows 10 PC.

[garyan2]

All correct.

The -b switch I added to the garbage collection is for WMC to create backup files for your tuner lineups, channel mappings, favorite lineups, custom channel names/numbers, and recording requests. In the event your database is corrupted and you experience a "recovery", the restored backups will be fairly recent to make the full recovery easier.

[Space]

Can you add a cache hit ratio? Might be helpful to determine if enabling the cache is worth the extra disk space needed.

Also, how does the image cache come in to play with If-Modified-Since? If the image is cached, does it just return that to WMC, or does it check if it has been modified on the SD server (and download it if it has changed) before returning the image?

Also, if the cache is turned off, does it just forward the "If-Modified-Since" header (if the request from WMC includes it) to the SD server?

[garyan2]

Can you add a cache hit ratio? Might be helpful to determine if enabling the cache is worth the extra disk space needed.

I'll see what can be done.

Also, how does the image cache come in to play with If-Modified-Since? If the image is cached, does it just return that to WMC, or does it check if it has been modified on the SD server (and download it if it has changed) before returning the image?

The latter. I throttle If-Modified-Since requests to 1 time every 12 hours for each image based on the last time it was verified. If the image has already been verified recently the service will just return either a 304 to the client or the image, depending on the requested I-M-S. If it has been more than 12 hours, then regardless if the client requested an I-M-S or not, and the image is cached, the service will send an I-M-S request to SD.

If the client sends an I-M-S request and the image does not exist in the cache, the service will forward the request to SD. If it is a 304 response, then that is just relayed back to the client. If the response contains the image, then that image will be added to the cache and sent to the client.

Also, if the cache is turned off, does it just forward the "If-Modified-Since" header (if the request from WMC includes it) to the SD server?

Yes. The service will not be a traffic cop either, so everything is just relayed back-and-forth with no intervention other than being the proxy.

[StinkyImp]

1) I've been using the portable version for years...

Thank you! You asked, and Gary answered, EVERY single question I had.

In fact, every post between your initial set of questions through this reply is absolutely fantastic!

A huge Thank You to everyone!

[IT Troll]

I just upgraded to v1.7.0.2 and now I do not have any images in my guide.

The service is running fine and there are no errors in trace.log. However, in server log I get multiple entries like this:

Code: Select all

[16/09/2021 13:36:53] /image/2a363f9198901a60b4f3ff388a241b269edc4eebb19197d13fa6c23bf2939fc6.jpg:
{"response":"UNKNOWN_USER","code":5004,"serverID":"20141201.web.1","message":"Download request from unknown account. Post message to http:\/\/forums.schedulesdirect.org\/viewforum.php?f=6","datetime":"2021-09-16T12:36:54Z"}

The account appears fine in the configuration client:

Code: Select all

[16/09/2021 13:35:51] ===============================================================================
[16/09/2021 13:35:51]  Activating the epg123 configuration GUI. version 1.7.0.2
[16/09/2021 13:35:51] ===============================================================================
[16/09/2021 13:35:52] [ INFO] Status request successful. account expires: 2022-07-23T10:51:28Z , lineups: 1/4 , lastDataUpdate: 2021-09-15T22:14:00Z
[16/09/2021 13:35:52] [ INFO] system status: Online , message: No known issues.

[garyan2]

Do you have the automatically refresh login enabled? You can open the configuration GUI to login to SD to refresh the timer on downloading images if for some reason that is not working.

[IT Troll]

Auto refresh is enabled. From the logs above you can see that the config GUI was opened just one minute before the image requests.

[garyan2]

I think I may have a gap here for new installs, will have to investigate later. Could you take a look in your registry at HKLM\SOFTWARE\GaRyan2\epg123 and make sure it exists? If it does, then stop and start the service from the configuration GUI and try to download images again.

One of the gaps I do have is that since the token is being stored in the registry now, and it will reuse the token if it is still valid, opening the GUI will no longer force a new token. Another thing you can do is login with a different machine which will force a new token issue on this machine when you open the gui because the "data" token will no longer be valid.

[IT Troll]

The registry existed and contained a token. Expiry was later tonight, ~24 hours after I upgraded to this version.

I restarted the service which then resulted in:

Code: Select all

[16/09/2021 17:43:25] Refreshed token upon receiving an UNKNOWN_USER (5004) error code.

Guide images are mostly appearing OK now. However, they are missing for some shows.

Code: Select all

[16/09/2021 17:43:25] /image/71d634446a438882e1f0df956fab67ccc70eb8f7ac50d942406c57c3c5685b67.jpg:
{"response":"IMAGE_NOT_FOUND","code":5000,"serverID":"20141201.web.1","message":"Could not find requested image. Post message to http:\/\/forums.schedulesdirect.org\/viewforum.php?f=6 if you are having issues.","datetime":"2021-09-16T16:43:25Z"}
[16/09/2021 17:43:26] /image/8649a51bea18f593f134f645189c87b3ff593d9c6931729e05eccdd32dd3facf.jpg:
{"response":"IMAGE_NOT_FOUND","code":5000,"serverID":"20141201.web.1","message":"Could not find requested image. Post message to http:\/\/forums.schedulesdirect.org\/viewforum.php?f=6 if you are having issues.","datetime":"2021-09-16T16:43:26Z"}
[16/09/2021 17:46:36] /image/0f492810243f844e76390e8a6e55786611231ec2e66b2d3b8118c17cada078f7.jpg:
{"response":"IMAGE_NOT_FOUND","code":5000,"serverID":"20141201.web.1","message":"Could not find requested image. Post message to http:\/\/forums.schedulesdirect.org\/viewforum.php?f=6 if you are having issues.","datetime":"2021-09-16T16:46:36Z"}

These are for quite well known shows which I am fairly sure had images previously. BTW, I don't currently have caching enabled (stand-alone full mode setup).