XML file structure causes errors

[ajocius1]

I have two tv stations that actually provide their program in xml file that they claim is xmltv compatible. I quickly tried to run XMLTVPDS but it gave me errors that I could not understand. I have then started comparing xml file I received from TV station to the example TVdata.xml in TVDataStarterKit. I found one clear difference that I think is causing the proglem. Channel ID and start time are in different order.

Here is example structure:
<programme channel="test24_ch" start="200407101100">

And here is what I get in xml file from tv channel:
<programme start="20070314063500" channel="LNK">

Is there any work arround to fix this problem? I know the only option to open that file in Excel, then manipulate colums, and then use ExcelPDS instead, but of course it would be nice to avoid that.

Any ideas?

[Jo]

I don't use XmlTvPDS but I looked in the code and as far as I can see the order is not important. Don't you get an error message?

[Bananeweizen]

Per XML definition the order of attributes in a tag doesn't matter. I don't know the XMLTV data structures, but I see another difference in the snippet: The time stamp includes seconds for LNK, while it doesn't for test24_ch.

[ajocius1]

Yes, I do get error message. It is big (around 230 kb) so it is possible too large to attach here. But this is how it looks inside:
-----------------------------------
java.io.IOException: Illegal time value: 'null'
at XmlTvPDS$TvDataHandler.extractDate(XmlTvPDS.java:477)
at XmlTvPDS$TvDataHandler.startElement(XmlTvPDS.java:290)
at org.apache.xerces.parsers.AbstractSAXParser.startElement(Unknown Source)
at org.apache.xerces.parsers.AbstractXMLDocumentParser.emptyElement(Unknown Source)
at org.apache.xerces.impl.dtd.XMLDTDValidator.emptyElement(Unknown Source)
at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanStartElement(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
at XmlTvPDS.parse(XmlTvPDS.java:143)
at XmlTvPDS.execute(XmlTvPDS.java:90)
at primarydatamanager.primarydataservice.AbstractPrimaryDataService.execute(AbstractPrimaryDataService.java:84)
at primarydatamanager.PDSRunner.PDSThreadRun(PDSRunner.java:118)
at primarydatamanager.PDSRunner.access$000(PDSRunner.java:14)
at primarydatamanager.PDSRunner$1.run(PDSRunner.java:73)
java.io.IOException: Illegal time value: 'null'
at XmlTvPDS$TvDataHandler.extractDate(XmlTvPDS.java:477)
at XmlTvPDS$TvDataHandler.startElement(XmlTvPDS.java:290)
at org.apache.xerces.parsers.AbstractSAXParser.startElement(Unknown Source)
at org.apache.xerces.parsers.AbstractXMLDocumentParser.emptyElement(Unknown Source)
at org.apache.xerces.impl.dtd.XMLDTDValidator.emptyElement(Unknown Source)
at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanStartElement(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
at XmlTvPDS.parse(XmlTvPDS.java:143)
at XmlTvPDS.execute(XmlTvPDS.java:90)
at primarydatamanager.primarydataservice.AbstractPrimaryDataService.execute(AbstractPrimaryDataService.java:84)
at primarydatamanager.PDSRunner.PDSThreadRun(PDSRunner.java:118)
at primarydatamanager.PDSRunner.access$000(PDSRunner.java:14)
at primarydatamanager.PDSRunner$1.run(PDSRunner.java:73)
-----------------------------------------------------

I have removed seconds manually. I opened file in a text editor, and replaced 00" with just ".

Here is also parto of xml file, if you guys can spot what needs to be changed:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE tv SYSTEM "xmltv.dtd">
<tv source-info-url="http://www.lnk.lt/programa/" source-info-name="LNK TV programa" generator-info-name="madEPG/0.2">
<channel id="LNK">
<display-name>LNK TV</display-name>
<url>http://www.lnk.lt</url>
</channel>
<channel id="TV1">
<display-name>TV1</display-name>
<url>http://www.tv1.lt</url>
</channel>
<programme start="200703150600" channel="LNK">
<title lang="lt">Dienos programa</title>
</programme>
<programme start="200703150605" channel="LNK">
<title lang="lt">DviraÄ

[ajocius1]

Forgot to mention, after I run XMLTVPDS, I do get files in Raw directory, but I also get those error messages. I tried to run datamanager and disregard errors, but then I got only 4 files with group name as part of the file name in prepared directory. No listings in other words.

[Jo]

It doesn't matter if there are additional digits at the end of the time stamps. It seems that the start date is missing twice somewhere. The exceptions are "Illegal time value: 'null'"

[ajocius1]

I only added small part of error message. The whole error message is veeeery long. Total error message file is 230 kb... I copied two paragraphs only, but there are hundreds of them and they all see the same. They alll clame "Illegal time value: "null'" . Seems like an error was in date, and that is How I cam to an idea, that perhaps channel id should be written first (prior) time stamp, which you both confirmed to be not correct....

I have also added part of xml file, the rest of the file looks exactly the same format. I could share the whole file, of course, if someone could have a look into it.

[Jo]

The code in XmlTvPDS is

Code: Alles auswählen

        private int extractTime(String value) throws IOException {
            // E.g. "200407101030"
            try {
                int hour = Integer.parseInt(value.substring(8, 10));
                int minute = Integer.parseInt(value.substring(10, 12));
                return hour * 60 + minute;
            }
            catch (Throwable thr) {
                throw new IOException("Illegal time value: '" + value + "'");
            }
        }

The exception "Illegal time value: 'null'" should mean that already the String "value" was null. But on the other hand as far as I can see, extractTime is only called if String value is not null. That means I don't know what's wrong. Maybe the developers do know.

[Jo]

What's the Starterkit you use? I refer to version 2.5.3. Maybe the code changed.

[ajocius1]

Jo, thank you for taking you time to help me.

Yes, I use version 2.5.3.

I wish I could debug program and see what is happening when it works with xml file, but my skills in java are equal to zero...

[Jo]

I mixed the functions up. The problem is certainly not in extractTime as I wrote above, but in extractDate as displayed in the error message.
And now I think i know the problem:
The data contains "<previously-shown />". XmlTvPDS needs something like "<previously-shown start="200703150605">" That means if the program is a repetition, the date and time of this repetition have to be in the data. It is the same with "next-time-shown" if the program will be repeated later.

[ajocius1]

Good News: You were right. I have removed all "<previously-shown />" from htm file and I got rid of that long error message. No it only complains that rating is not a number. An example of error message:
Warning: rating is no number: 'N7', and will be ignored.

Well I don't mind that to be ignored, so I hoped it was the last problem I faced. However, running datamanager I still got only four files in prepared directory. The message on DOS screen looked as follows:
----------------------------
Fine: Updating the channel list
Fine: Updating the mirror list
writing summery to the file...
get the mimimum start date...
write frames...
frameCount: 0
minStartDaysSince1970: 2147483647
done.
-----------------------------

and no file with tvguide listings...

[Jo]

Try "dayprogramfiletranslator raw" first. Then you can see the data in text files in /raw/txt. Maybe you can see there what's wrong.

[ajocius1]

dayprogramfiletranslator raw did not give any errors. The output files seems to be fine as far as I understand. Each file has date and channel name in it's name. Example "2007-03-19_de_LNK_raw_full.prog.txt". Text inside looks like:

Version: 1

Program ID: 0
start time: 6:00
original title: Dienos programa

Program ID: 1
start time: 6:05
original title: Dvira?io šou
description: Humoro laida.
Ved?jai Kristina Savickyt? ir Vytautas Šer?nas.

Program ID: 2
start time: 6:35
original title: Nuostabiausi mitai ir legendos (12)
description: (Wonderous Myths & Legends).
production year: 1999
genre: Animacinis serialas
origin: JAV

Program ID: 3
start time: 7:00
original title: Visos žvaigžd?s (24)
description: (Baby Looney Tunes I). Rež.: Michael Hack, Scott Heming.
production year: 2002
genre: Animacinis serialas
origin: JAV

Program ID: 4
start time: 7:25
original title: Nekviesta meil? (44)
description: Rež.: Alvydas Šlepikas. Vaid.: Inga Jankauskait?, Adolfas Ve?erskis, Rimant? Valiukait?, Saulius Balandis.
production year: 2006
genre: Serialas
origin: Lietuva

Program ID: 5
start time: 8:00
original title: Langai III (617)
description: (Okna III).
production year: 2005
genre: Pokalbi? laida
origin: Rusija

Above is not complete file, i just copied first 5 programs and I didn't see structural changes in following programs. I also see there ir a log file called datamanager.log.0, but info inside does not lead me anywhere:
<?xml version="1.0" encoding="windows-1252" standalone="no"?>
<!DOCTYPE log SYSTEM "logger.dtd">
<log>
</log>
<?xml version="1.0" encoding="windows-1252" standalone="no"?>
<!DOCTYPE log SYSTEM "logger.dtd">
<log>
</log>


Is there any clever way to debug the program without much knowledge about java?

[Jo]

Is there any clever way to debug the program without much knowledge about java?

You can add some "System.out.println(variable);"
I don't know what's wrong. I don't really know how XmlTvPDS works. Maybe you can find out what's different on those days that work. Or try it without the rating element. Even if it says "WARNING" it might be an error.