Reading Tdms files

[NathanHuynh]

I am running a script to read through the tdms files. However, I have run into a two problems.
Running the script through one folder is fine. However, when I run the script through a sub-directory of folders I run into two errors which I cannot find out what it means.
I have the loop read files only ending with '.tdms' within the sub-directories.

1). ValueError: Segment does not start with TDSm, but with $ '

2). ValueError: Segment does not start with TDSm, but with " %

[adamreeve]

It sounds like you are trying to read files that are not valid TDMS files, as they don't start with the first four TDSm bytes that should be at the beginning of all TDMS files.

[adamreeve]

I'm closing this issue but please reopen if you can reproduce this error with valid TDMS files.

[SeyedParvasi-NOV]

I've got the same issue trying to open a large TDMS file:

ValueError: Segment does not start with b'TDSm', but with b'\xb1\x92\xaf\xf1'

Anyone know how to resolve this issue?

[adamreeve]

Hi @SeyedParvasi-NOV, if this is a valid TDMs file that LabVIEW can read then that sounds like a bug in npTDMS. Are you able to provide an example file that reproduces the problem?

[ghost]

Hi, i've got the same issue, i looked up the tdfm file in editor and it starts with 'TDSm'

Anyone has already solved this issue?

[adamreeve]

Hi @altgynr, I'm happy to take a look at this issue if you can provide an example file with this problem.

[Joe-Heffer-Shef]

Are you opening the .tdms_index file instead of the .tdms file?

[Eifi1]

.tdms_index files should start with b'TDSh' to be a valid file. Therefore that might not be the root cause.

[Tsopalsk]

Hi Guys,

i am running into the same problem.
But it occurs only, when i run my python script via crontab in debian/linux.

Concrete error:
"the error Segment does not start with b'TDSm', but with b'<!DO' occured"

fails at line 3:
from nptdms import TdmsFile
FilePath = r"/home/user/tdms2/cts2.tdms"
tdms_file = TdmsFile.read(FilePath) //this step does fail

What i don't understand is, when i run the code directly from my directory,where the script is located, everything is working fine.
So i assume, the .tdms file i am using is valid, it works pretty much with every .tdms file i have, but only on the manual execution.

I can even reproduce the error on the manual execution if i paste an absolute path that does not exist, e.g:
FilePath = r"/home/user/tdms2/c2.tdms"

error will be:
"the error Segment does not start with b'TDSm', but with b'<!DO' occured"

When i change it back it works again on the manual execution.

Does anyone have an idea, why crontab makes the script fail with this error?

[Joe-Heffer-Shef]

Hi Guys,

i am running into the same problem. But it occurs only, when i run my python script via crontab in debian/linux.

Concrete error: "the error Segment does not start with b'TDSm', but with b'<!DO' occured"

fails at line 3: from nptdms import TdmsFile FilePath = r"/home/user/tdms2/cts2.tdms" tdms_file = TdmsFile.read(FilePath) //this step does fail

What i don't understand is, when i run the code directly from my directory,where the script is located, everything is working fine. So i assume, the .tdms file i am using is valid, it works pretty much with every .tdms file i have, but only on the manual execution.

I can even reproduce the error on the manual execution if i paste an absolute path that does not exist, e.g: FilePath = r"/home/user/tdms2/c2.tdms"

error will be: "the error Segment does not start with b'TDSm', but with b'<!DO' occured"

When i change it back it works again on the manual execution.

Does anyone have an idea, why crontab makes the script fail with this error?

<!DO looks like the beginning of a HTML document.

https://www.w3schools.com/tags/tag_doctype.asp

Can you post the error message?

[Tsopalsk]

Hi,
this is the full error message from debian mail log:

the error Segment does not start with b'TDSm', but with b'<!DO' occured
Traceback (most recent call last):
File "/home/user/tdms2/tdms2.py", line 34, in job
tdms_file = TdmsFile.read(FilePath)
File "/home/user/.local/lib/python3.7/site-packages/nptdms/tdms.py", line 69, in read
return TdmsFile(file, raw_timestamps=raw_timestamps, memmap_dir=memmap_dir)
File "/home/user/.local/lib/python3.7/site-packages/nptdms/tdms.py", line 130, in init
self._read_file(self._reader, read_metadata_only, keep_open)
File "/home/user/.local/lib/python3.7/site-packages/nptdms/tdms.py", line 226, in _read_file
tdms_reader.read_metadata(require_segment_indexes=keep_open)
File "/home/user/.local/lib/python3.7/site-packages/nptdms/reader.py", line 88, in read_metadata
file, segment_position, index_cache, previous_segment, reading_index_file)
File "/home/user/.local/lib/python3.7/site-packages/nptdms/reader.py", line 236, in _read_segment_metadata
file, segment_position, is_index_file)
File "/home/user/.local/lib/python3.7/site-packages/nptdms/reader.py", line 254, in _read_lead_in
"Segment does not start with %r, but with %r" % (expected_tag, tag))
ValueError: Segment does not start with b'TDSm', but with b'<!DO'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/user/tdms2/tdms2.py", line 103, in
job()
File "/home/user/tdms2/tdms2.py", line 98, in job
server.login(mailuser, mailpw)
File "/usr/lib/python3.7/smtplib.py", line 721, in login
initial_response_ok=initial_response_ok)
File "/usr/lib/python3.7/smtplib.py", line 638, in auth
authobject(challenge).encode('ascii'), eol='')
AttributeError: 'NoneType' object has no attribute 'encode'

[adamreeve]

I can even reproduce the error on the manual execution if i paste an absolute path that does not exist, e.g:
FilePath = r"/home/user/tdms2/c2.tdms"
error will be:
"the error Segment does not start with b'TDSm', but with b'<!DO' occured"

This seems quite odd, you should just get a FileNotFoundError exception. Are you reading these files from a normal file system or is it some kind of FUSE based file system that is returning HTML data for paths that don't exist perhaps?

When run as a cron job, the process will run in a different environment than your usual user shell. Are you using absolute paths in your cron job too or does the path depend on the environment somehow? https://stackoverflow.com/tags/cron/info and https://serverfault.com/questions/449651/why-is-my-crontab-not-working-and-how-can-i-troubleshoot-it might be a good place to start with debugging your cron job.

[Tsopalsk]

I had some time to look into this issue again.
You were right about the different environment, that crontab uses.

I could fix it, by importing my User Environment from the Debian into the crontab script.
Solution looks like this:
SHELL=/bin/bash
PATH=/home/user/.local/bin:/usr/local/bin
* * * * * . $HOME/.profile; /usr/bin/python3 /home/user/script1.py

Thanks for the help!