Skip to content

Commit

Permalink
improved Readme, moved warnings to stderr
Browse files Browse the repository at this point in the history
  • Loading branch information
Yves Ulrich Tittes committed Jan 22, 2025
1 parent 36d618f commit 3b2f713
Show file tree
Hide file tree
Showing 5 changed files with 24 additions and 12 deletions.
3 changes: 2 additions & 1 deletion .goreleaser.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,8 @@ archives:
- goos: windows
format: zip
files:
- tutorial/**
- LICENSE
- README.md

changelog:
sort: asc
Expand Down
8 changes: 4 additions & 4 deletions LS_Metadata_reader/metadata_parser.go
Original file line number Diff line number Diff line change
Expand Up @@ -509,7 +509,7 @@ func startProgressReporter(progressTracker *ProgressTracker) {
progress := float64(completed) / float64(total) * 100
fmt.Printf("\rProgress: %.2f%%", progress)
if completed >= total {
fmt.Printf("\n")
fmt.Println("")
break
}
time.Sleep(100 * time.Millisecond)
Expand Down Expand Up @@ -561,11 +561,11 @@ func Reader(directory string, zFlag bool, fFlag bool, p3Flag string) ([]byte, er
var getmpc map[string]string
config, err := configuration.Getconfig()
if err != nil {
fmt.Println("Warning: No path config available, we suggest using either --epu or the config to provide the path where EPU mirrors the datasets and stores xmls")
fmt.Fprintln(os.Stderr, "Warning: No path config available, we suggest using either --epu or the config to provide the path where EPU mirrors the datasets and stores xmls")
} else {
errun := json.Unmarshal(config, &getmpc)
if errun != nil {
fmt.Println("Your config was unretrievable, make sure it is set and accessible or use the param flags")
fmt.Fprintln(os.Stderr, "Your config was unretrievable, make sure it is set and accessible or use the param flags")
}
parallel = getmpc["MPCPATH"]
}
Expand All @@ -576,7 +576,7 @@ func Reader(directory string, zFlag bool, fFlag bool, p3Flag string) ([]byte, er
if parallel != "" {
dataFolders, err = findDataFolders(parallel+target, dataFolders)
if err != nil {
fmt.Println("There should be a folder on your instrument control computer with the same name - something went wrong here", err)
fmt.Fprintln(os.Stderr, "There should be a folder on your instrument control computer with the same name - something went wrong here", err)
return nil, err
}
}
Expand Down
14 changes: 10 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# LS Metadata extractor
simple parser for the two common life-science EM metadata output formats, written in go
simple parser for the two common life-science EM metadata output formats (.xml from EPU and .mdoc from TOMO5 and SerialEM respectively), written in go

## Usage
Chose the appropriate binary from the [Releases](https://github.com/SwissOpenEM/LS_Metadata_reader/releases), then:
LS_reader_Version target_directory
LS_Metadata_reader target_directory

For testing, try the associated tutorial folder; an example of how the output should look like is provided in the same folder (tutorial_correct.json). For first time use disregard the warnings about config/flags those are for use directly with EPU or the OpenEM Ingestor.
For testing, try the associated [tutorial](https://github.com/SwissOpenEM/LS_Metadata_reader/tree/main/tutorial) folder; an example of how the output should look like is provided in the same folder (tutorial_correct.json).

## Comments
Runs on a directory containing raw files and their instrument written additional information files (.mdoc and .xml respectively), generates a dataset level .json file. In case of usage with EPU pointing to the top level directory is enough, it will search for the data folders and extract the info from there. Using --z you can also obtain a zip file of the xml files associated with your data collection. If you want all the metadata (dataset level, not all individual entries) written out by a given software use the --f flag, otherwise the output will be OSCEM conform.
Expand All @@ -15,7 +15,13 @@ SerialEM properties examples are to be added to the existing properties files of
**!!! Requires SerialEM 4.2.0 or newer !!!**

## For running with EPU directly
Benefits from setting the "MPCPATH" variable in the .config file (set using LS_reader_version --c) to define the path of data acquistisions mirrored on the microscope PC in EPU. Will work regardless if pointed to the xmls/mdocs otherwise.
Benefits from setting the config (set using LS_Metadata_reader --c), or handing over the three instrument values directly with flags:
--cs
--gain_flip_rotate
--epu
The --cs (for the CS value of the instrument) and --gain_flip_rotate (for the orientation of the gain_reference relative to actual data) are unfortunately never provided in the metadata, and are both important for processing. It is therefore highly beneficial to set these two.
As for --epu, EPU writes its metadata files in a different directory than its actual data (TOMO5 also keeps some additional info that is processed by the LS_Metadata_reader there). It generates another set of folders, usually on the microscope controlling computer, that mirror its OffloadData folders in directory structure. Within them it stores some related information, among which are also the metadata xml files. If --epu is defined as a flag or in the config, the LS_Metadata_reader will directly grab those when the user points it at a OffloadData directory.
NOTE: This requires you to mount the microscope computer directory for EPU on the machine you are running LS_Metadata_reader on as those are most likely NOT the same. The extractor will work regardless if pointed to the xmls/mdocs directly, this is just for convenience.

## Schema-Links
Output is compatible to OSCEM schemas https://github.com/osc-em/OSCEM_Schemas/
Expand Down
7 changes: 5 additions & 2 deletions conversion/converter.go
Original file line number Diff line number Diff line change
Expand Up @@ -156,11 +156,14 @@ func Convert(jsonin []byte, content embed.FS, p1Flag string, p2Flag string) ([]b
if p1Flag == "" && p2Flag == "" {
config, err := configuration.Getconfig()
if err != nil {
fmt.Println("Warning: config was not set or could not be obtained - make sure the config is set using LS_Metadata_reader --c or you are using the flags")
_ = err
//could enable the warning below but its not really required
//fmt.Fprintln(os.Stderr, "Warning: config was not set or could not be obtained - make sure the config is set using LS_Metadata_reader --c or you are using the flags")
} else {
errun := json.Unmarshal(config, &fixvalues)
if errun != nil {
fmt.Printf("%s", "Warning: Config was found but unreadable")
_ = errun
//fmt.Fprintln(os.Stderr, "Warning: Config was found but unreadable")
}
}
} else {
Expand Down
4 changes: 3 additions & 1 deletion main.go
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ func main() {
cut := strings.Split(cwd, string(os.PathSeparator))
name := cut[len(cut)-1] + ".json"
os.WriteFile(name, output, 0644)
fmt.Println()
fmt.Println("Extracted data was written to: ", name)

} else {
Expand All @@ -86,6 +87,7 @@ func main() {
twd = strings.Join(conc, ".")
}
os.WriteFile(twd, output, 0644)
fmt.Println("Extracted data was written to: ", twd)
fmt.Println()
fmt.Printf("Extracted data was written to: %s", twd)
}
}

0 comments on commit 3b2f713

Please sign in to comment.