Inconsistent scaling in 'Agilent' .ch files
#30
Comments
|
Hmm, I reran again in a restarted RStudio environment, removing the data scaling ( But all sorted now! Feel free to keep the .ch if it helps at all, I'll leave the repository up. |
|
Hi Tom, Thanks for checking in about this and providing the test files. I suspect this may be the same issue addressed in #22 -- it seems that MassHunter ignores the scaling factor for some ChemStation files, but when I opened the file in ChemStation it displays the scaled values. In terms of actual solutions, I could add a "scale" argument to the "read_chemstation_ch" function to toggle the scaling if that would be helpful? I think I would keep the scaled ChemStation values as the default since the data is generated in ChemStation. Best, |
|
Hi Ethan, Thanks for getting back to me this quickly. Apologies for bringing up something thats already been discussed! A scale toggle would be really helpful. For the work I'm going through at the moment I just saved a local version without the data scaling operation, but yeah having it as a parameter directly in the function call would be great. Interesting that MassHunter and ChemStation like this, don't really see the reason for it to be honest. Thanks again! :) Cheers, |
|
No worries! And yes, it is rather perplexing. I will add the toggle to the next update probably in the next couple of days. Will update you here |
|
After taking a closer look at your file I think you’re right that’s something is off with the scaling factor in your file. The values shouldn’t be on the scale of 10^-300. I’m going to put your file in chemstation and see how it looks in there. |
|
Yeah its odd because it seems to be the standard location for scaling factors in literally any other 179 .ch file I've looked at other than those from our lab. As I say though the actual values themself aren't scaled in this file. Now I'm curious what it is at the 0x127c location for our files! These .ch files and .ms files are automatically synced with our integration software, which isn't anything to do with Agilent - could be the reason ours isn't scaled? Although I'm really clutching at straws! |
|
So I tried to open your file in my version of ChemStation (Rev. B.04.02 SP1 [212]) ) and it's just showing a blank plot, so that doesn't really resolve anything, but kind of interesting. Maybe it is applying the scaling factor and getting confused because the values are too small. |
|
Yeah maybe? I've just tried opening in OpenChrom and it gives the 'unscaled' values, ie ~2E 4 (for both importing as .xy and .ocb).
Just out of curiosity I opened up the attached FID file from #22 in OpenChrom too and it also gave the 'unscaled' values, ie what it is showing in MassHunter.
Do you have access to other 179 .ch files? I could try opening them in OpenChrom too to see which values it imports. Could it be there is something within the .ch file which informs whether there is scaling or not? |
|
Ya, it's definitely possible... There are still some values in the headers that I don't know what they are. I have a small collection of 179 files that I've accumulated from various sources. You're welcome to take a look! (https://cornell.box.com/s/yfsiaav9puyug5y39dqekealtujd8gfu). Let me know if you can't access them through the Box link for some reason. |
|
Awesome, cheers! I'll have to take a look tomorrow, it's nearly 10pm here in the UK haha. Thanks again! 😊 |
|
I added a |
|
So I opened up the FID1A.ch file from 8byte > mustang > rainbow_yellow.D into OpenChrom, and it produced this chromatogram:
This file offers the same scaling factor of ~0.00013 as in #22 , so is scaled accordingly in
|
|
So I have "Asterix". I guess one possibility is that the file from "Mustang" Chemstation are unscaled and my ChemStation is incorrectly interpreting them by adding the scaling factor. You don't by any chance have access to a Mustang Chemstation box you could check on do you? |
|
Ah I'm afraid not - I've tried to hunt around in the MassHunter acquisition software but can't find anything about data scaling in the file outputs. Tbh I didn't know there was a version Asterix - I only knew about Mustang (GCMS) and Leonardo (GC). If you have a .ch for a sample you know is scaled using the scaling factor in the .ch file, could you send it over? I can try run it in OpenChrom, and if you have the .D file for it then I can try open it in MassHunter Qualitative (that opens MS from a .D first and then extracts non-MS chromatograms so needs the full .D file). |
|
Hi Tom, I just checked out the intensities of all the ChemStation test files again (I can't open the Open Lab files). My version of ChemStation displays both of the Mustang files with the scaled intensities. Whether that's correct or not I can't say for sure. I noticed that the Asterix files are actually scaled incorrectly by chromConverter compared to how they appear in ChemStation, however the unscaled version is also wrong. Maybe the scaling factor has a different encoding in these files...? Unfortunately, I already shared with you everything that I have as far as 179 files go. I don't have the full .D folders for most of those test files. I've attached screenshots of the incorrectly scaled chromatograms from the Asterix files.
Ethan |
|
Tbh it was only from one of the guys at the lab that told me about Leonardo - I think its only for GC acquisition, but I'm really not too sure. All of the files from our lab seem to be Mustang. But, found some possibly cool stuff! If you extract the zip I've attached, and load the FID1A.ch into 20240419_00177.D.zip |
|
Very interesting. So you are acquiring your .CH files in MassHunter? As far
as I know, .D files from ChemStation don't contain this AcqData folder. I
wonder if disparities between mass hunter and chemstation could be the root
of some of these scaling issues? The "Yellow.D" folder also seems to be
acquired in MassHunter?
…On Fri, Aug 16, 2024 at 7:11 AM TomWarburtonIA ***@***.***> wrote:
Tbh it was only from one of the guys at the lab that told me about
Leonardo - I think its only for GC acquisition, but I'm really not too
sure. All of the files from our lab seem to be Mustang.
But, found some possibly cool stuff! If you extract the zip I've attached,
and load the FID1A.ch into read_chemstation_ch without scaling, you get
the dataframe which I've attached below
('extracted_RT_and_unscaled_intensities.csv'). However, if you go into the
20240419_00177.D folder, and then into AcqData, and load both FID1.cd and
FID1.cg into a hex editor (I use HxD), and make sure the byte order is
little endian. First go to FID.cd and offset 0x9E, you find that after the
intensity unicode characters, you get a number 1. And then read FID.cg
into a hex editor, you can literally scroll through all of the intensities
if you stay on the 4th byte of the row (changing to 8 bytes per row helps
as you can just press the down arrow and go through them all). So I think
that the .ch file is probably a composite of the stuff in AcqData.
Obviously I can't speak for it all as I only know our files, but its
possible the scaling for our stuff is held in the FID[number].cd file. I
checked for the rainbow_yellow.D too and they have the same number 1 at
offset 0x9E in their FID1.cd file. Interesting stuff!
20240419_00177.D.zip
<https://github.com/user-attachments/files/16636289/20240419_00177.D.zip>
extracted_RT_and_unscaled_intensities.csv
<https://github.com/user-attachments/files/16636291/extracted_RT_and_unscaled_intensities.csv>
—
Reply to this email directly, view it on GitHub
<#30 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADZEBO7XUOIZ2EDG2PYPVLDZRXM5HAVCNFSM6AAAAABMMDIDPWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJTGMYTANRSGQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
Yeah our data is acquired through MassHunter. Won't know without seeing more of these data files, but I think this is probably the cause of the scaling issue when using a MassHunter-acquired chromatogram. I'm not sure if there is a way to discern it specifically from the .ch file, so perhaps its best left down to the user to determine if the data has been scaled or not if using |
|
Ya, I'm also not 100% convinced though it certainly seems like a good hypothesis! I think for now I will just leave it up to the user to toggle the scale factor and maybe add a note to the documentation that CH files generated by masshunter may be unscaled. Thanks for looking into this! (and certainly happy to keep the conversation going if you make any further discoveries). |
|
Yeah probably for the best to be left up to the user - I think its fair to assume a certain level of understanding of their .ch files, and if the scaled values seem to be inaccurate then its just a simple logical to change in the code when calling No problem! It wouldn't be nearly as difficult if Agilent was at least consistent with data acquisition and documentation methods, but maybe thats the point! |
ethanbass
changed the title
.ch files
|
Yes, it would certainly be nice!! I will close this for now, but feel free to reopen if anything comes up in connection with this! |
Hi Ethan,
Firstly, chromConverter is awesome! Thanks for all your work on this - its brilliant to be able to read these proprietary files into R. I've had a slight issue with importing my .ch files, in that the scaling for the intensity doesn't seem to be correct. Opening in Agilent MassHunter gives intensities around 2E 4, and the unscaled values appear to be something around 6.66 (using your code for
read_chemstation_chand bypassing the data transformation linedata <- data * scaling_factor + intercept). I've attached a link to a repository with the .ch file along with the two data frames showing scaled and unscaled intensities (can't attach .ch directly here). The usual location of the scaling factor in a 179 .ch file (which this is) doesn't appear to be correct for this .ch? I believe it is usually 0x127c, but in this .ch file from our instrument is doesn't appear to be so?I must admit to being a bit out of my depth with interpreting this though! I'm not all too confident navigating it in hex editors etc. For what its worth, the relative intensities are all correct (ie. its the correct chromatogram - peaks all look as they should), but not too sure where the scaling factor actually is.
I don't really need to do much analysis for this all being told - it was just to input it into R so I can apply themes to the chromatogram so it is in keeping with other plots I am making. So I could just scale the intensities from 0 to 100 and make it relative. But if you were able to offer any guidance about how to get the scaling correct then I'd be really grateful
Cheers!
Tom
Attachments:
https://github.com/TomWarburtonIA/agilent_ch
The text was updated successfully, but these errors were encountered: