Skip to content

Commit

Permalink
Merge in removal of submodule references and replacement with copies …
Browse files Browse the repository at this point in the history 
…of the DataTree and Utils libraries. Submodules too difficult to work with and bad git support (#107) (#109)

* v1.32.4

-- Fix some bugs in path utils functions
-- More improvements in build/installation process

* Update probe class to save and load 3D landmarks

* Update probe class to work with old SD files

* v1.32.4

-- Copy versoin number for Homer3 from AV

* -- Fix small bug in setpaths(0) paths removal

* CCA shrinkage back on

* Update hmrR_PreprocessIntensity_Negative.m

quick fix on negative value check

* v1.32.5

-- Fix issue with tCCA user functions not being loaded into function registry
-- Set data tree structure display in MainGUI to include the group name display unconditionally. In some instances the group name did not show if the all subjects had only one run.

* Update hmrR_PreprocessIntensity_Negative.m

Function to include zero values.

* v1.32.5

-- Sync DataTree and Utils with AtlasViewer

* v1.32.7

-- Add more robust error checking to exclude .snirf files that can't load DataClass.m
-- Add better error checking to DataClass and MeasListClass

* Create shared code and submodule references to DataTree and Utils (#74)

* -- Move DataTree and some Utils functions to independent submodules used by AtlasViewer and Homer3

* Add config file to DataTree and ability to load mutiple config files to config utility

Add config file to DataTree and ability to load mutiple config files to config utility

* Relink submodule references from jayd1860 to BUNPC

* Fix submodule download not working on linux when initializing repo with setpaths. Needed to disable git certificate verification when executing 'git submodule update' from matlab

* -- Add tool to setpaths to be able to download submodules without having git installed.

* -- Change code to setpaths not to initialize submodules every toime it is run. Add option to 'update' submodules to get latest changes.

* ISSUE_TEMPLATE folder (#77)

* Fix setpaths issue on MAC where the initialization of submodules causes setpaths to hang waiting for user to press ENTER because the system command running 'git branch' executes in its own shell. Setting the TERM=ansi seems to fix this issue.

* -- Add .gitignore file to simplify and clatrify git status output

* -- In previous commit didn't entirely fix issue with setpaths waiting for user input on MAC. Need to parse and remove garbage output correctly too

* Just use Snirf style GetAux to display aux data (#75)

* -- Set origin of submodules to be same as parent repo to follow paradigm of submodules and parent repo being one code (#78)

-- Fix submodule download in setpaths without git not working because setpaths options are not parsed correctly.
-- Clean up and remove obsolete files

* -- Fix issue in setpaths when attempting to download submodules using the non-git method. Assumtion that Github creates empty folder as placeholders for submodules when downloading using the "Download ZIP" option is not always true. Sometimes it creates empty folder, somtimes it doesn't. So isemptyFolder doesn't work when folder is not created. Added code to create empty submodule folder placeholder when it is missing.

-- Clean up: remove more obsolete files

* -- Fix typo

* -- Fix issue with setpaths submodules when there's a partial submodules download because of a previous error then need to clean uo submodule folders so there's no garbage in there then retry downloading.

* -- Greatly simplify and make it more straighfoward the setpaths process.
-- Add .numberfiles to submodules and check during setpaths process to make sure an incomplete download is detected and triggers a clean download.

* -- Fix setpaths error exception on linux and mac because ls([pathname, '*.m']) works differently on those platform than on Windows when there are NO files matching the specified pattern. Use dir instead of ls to fix this.
-- Fix multiple issues in submodule downloading using non-git method.
-- When user click cancel in response to download fail dialog, quit rather than retry.
-- Add search paths ONLY for the submodules that were successfully installed
-- Fix typos in dialog failed download message

* -- a) Fix setpaths bug in removeFolderContents.m on linux and MAC because the delimiter for path string is ':' not ';' as it is on Winodws. b) On linux the gitSetBranch.m used to get latest branch revis

* Fix setpaths('update') not working.

* -- Fix problems in setpaths with creating new submodule branches

-- Fix issue in setpaths with non-git submodule download and install.
   a) Fix some bugs and typos preventing correct download.
   b) Clean up and delete downloaded submodules files which are no longer needed after installation like the zip file and initial unzipped folder
   c) Clarify input dialog message and try to guess and or suggest the correct branch that matches parent repo

-- Add exception handling to setpaths. in case of error return to original folder

-- Add comprehensive help comments to setpaths describing it's use

* -- Don't add current app twice. Check app include list to see if it already has been added to search paths

* Glm fixes (#80)

- GLM docs fixes

- Convolve HRFs with stim duration before export

* -- In setpaths exclude more folders to avoid problems and headaches with unexpected matlab behavior because of inclusion of weird .m files from a build of AV. So for instance exclude *.app and *_install folders which contain a large amount of files which should never be included in matlab search paths.
-- Add ability to findDotMFolders to exclude folders based on wildcard patterns rather than only explicit names. This allows exclusion of many folders that fit a pattern without having to explicitly list ALL of them
-- Fix several bugs in the build for MAC
-- Finally fixed the requirement that you have to run MAC installation specifically from ~/Downloads/homer3_install. Added code to makesetup.pl to create a more intelligent setup.command which does not depend on execution from a specific folder.

* -- Fix build not working because need to copy ALL config files to installation folder
-- Improve launch performance by changing config file access to global variable

* -- Change setpath submodule download without git, to not ask questions about which branch to download. Instead derive the branch name from root folder name of parent app and default to master of it cannot be determined. This change somehow did not make it earlier into repo

* -- Update submodule references to latest development branch rev

* -- Change unit test to use global config variable rather than opening/closing ConfigFileClass objects
-- Update getVernum to match AV
-- Update submodule references

* DataTree submodule reference update and Mac support fixes (#89)

-- Update DataTree submodule reference to latest change containing fix for errors when there are no data files (because all have errors that prevent loading)

-- Fix more issues with install executable on MAC

* Gets info about latest version from GitHub API rather than openfnirs.org (#92)

* Gets info about latest version from GitHub API rather than openfnirs.org

* Version++

* -- Change setpaths to not download submodules by default. Use 'init' or 'update' options to download automatically. If using github desktop then github desktop download submodules automatically. (#95)

-- Change downloadSharedLibs to change .gitmodules origin url to point to where ever parent came from
-- Add .gitattributes to exclude files from branch merges such as submodule references and .gitmodules
-- Sync getVernum with Homre3. (getVernum will eventually be moved to Utils/Shared library)
-- Improve creation of installation file and installation process using myCopyFile.m

* -- Update submodule reference for development branch to latest
-- Fix .gitmodules to point to development branch instaead of master

* -- Fix setpaths gitSubmodulesUpdate not updating changes to submodules correctly

* -- Fix wrong path to DataTree to exclude from merges
-- Add .gitattributes in list of files to exclude fropm merges

* -- Update submodule references

* -- Update submodule reference to Utils

* Remove submodule references and replace with copies of the DataTree and Utils libraries. Submodules too difficult to work with and bad git support (#107)

* -- Update DataTree reference pointer

* -- Update Utils submodule reference to latest

* Remove submodules: too hard to work with, not enough support from git

* Remove submodules: too hard to work with, not enough support from git

* Merge in change to get rid of submodule references which are more trouble than they are worth (#2)

* v1.32.4

-- Fix some bugs in path utils functions
-- More improvements in build/installation process

* Update probe class to save and load 3D landmarks

* Update probe class to work with old SD files

* v1.32.4

-- Copy versoin number for Homer3 from AV

* -- Fix small bug in setpaths(0) paths removal

* CCA shrinkage back on

* Update hmrR_PreprocessIntensity_Negative.m

quick fix on negative value check

* v1.32.5

-- Fix issue with tCCA user functions not being loaded into function registry
-- Set data tree structure display in MainGUI to include the group name display unconditionally. In some instances the group name did not show if the all subjects had only one run.

* Update hmrR_PreprocessIntensity_Negative.m

Function to include zero values.

* v1.32.5

-- Sync DataTree and Utils with AtlasViewer

* v1.32.7

-- Add more robust error checking to exclude .snirf files that can't load DataClass.m
-- Add better error checking to DataClass and MeasListClass

* Create shared code and submodule references to DataTree and Utils (#74)

* -- Move DataTree and some Utils functions to independent submodules used by AtlasViewer and Homer3

* Add config file to DataTree and ability to load mutiple config files to config utility

Add config file to DataTree and ability to load mutiple config files to config utility

* Relink submodule references from jayd1860 to BUNPC

* Fix submodule download not working on linux when initializing repo with setpaths. Needed to disable git certificate verification when executing 'git submodule update' from matlab

* -- Add tool to setpaths to be able to download submodules without having git installed.

* -- Change code to setpaths not to initialize submodules every toime it is run. Add option to 'update' submodules to get latest changes.

* ISSUE_TEMPLATE folder (#77)

* Fix setpaths issue on MAC where the initialization of submodules causes setpaths to hang waiting for user to press ENTER because the system command running 'git branch' executes in its own shell. Setting the TERM=ansi seems to fix this issue.

* -- Add .gitignore file to simplify and clatrify git status output

* -- In previous commit didn't entirely fix issue with setpaths waiting for user input on MAC. Need to parse and remove garbage output correctly too

* Just use Snirf style GetAux to display aux data (#75)

* -- Set origin of submodules to be same as parent repo to follow paradigm of submodules and parent repo being one code (#78)

-- Fix submodule download in setpaths without git not working because setpaths options are not parsed correctly.
-- Clean up and remove obsolete files

* -- Fix issue in setpaths when attempting to download submodules using the non-git method. Assumtion that Github creates empty folder as placeholders for submodules when downloading using the "Download ZIP" option is not always true. Sometimes it creates empty folder, somtimes it doesn't. So isemptyFolder doesn't work when folder is not created. Added code to create empty submodule folder placeholder when it is missing.

-- Clean up: remove more obsolete files

* -- Fix typo

* -- Fix issue with setpaths submodules when there's a partial submodules download because of a previous error then need to clean uo submodule folders so there's no garbage in there then retry downloading.

* -- Greatly simplify and make it more straighfoward the setpaths process.
-- Add .numberfiles to submodules and check during setpaths process to make sure an incomplete download is detected and triggers a clean download.

* -- Fix setpaths error exception on linux and mac because ls([pathname, '*.m']) works differently on those platform than on Windows when there are NO files matching the specified pattern. Use dir instead of ls to fix this.
-- Fix multiple issues in submodule downloading using non-git method.
-- When user click cancel in response to download fail dialog, quit rather than retry.
-- Add search paths ONLY for the submodules that were successfully installed
-- Fix typos in dialog failed download message

* -- a) Fix setpaths bug in removeFolderContents.m on linux and MAC because the delimiter for path string is ':' not ';' as it is on Winodws. b) On linux the gitSetBranch.m used to get latest branch revis

* Fix setpaths('update') not working.

* -- Fix problems in setpaths with creating new submodule branches

-- Fix issue in setpaths with non-git submodule download and install.
   a) Fix some bugs and typos preventing correct download.
   b) Clean up and delete downloaded submodules files which are no longer needed after installation like the zip file and initial unzipped folder
   c) Clarify input dialog message and try to guess and or suggest the correct branch that matches parent repo

-- Add exception handling to setpaths. in case of error return to original folder

-- Add comprehensive help comments to setpaths describing it's use

* -- Don't add current app twice. Check app include list to see if it already has been added to search paths

* Glm fixes (#80)

- GLM docs fixes

- Convolve HRFs with stim duration before export

* -- In setpaths exclude more folders to avoid problems and headaches with unexpected matlab behavior because of inclusion of weird .m files from a build of AV. So for instance exclude *.app and *_install folders which contain a large amount of files which should never be included in matlab search paths.
-- Add ability to findDotMFolders to exclude folders based on wildcard patterns rather than only explicit names. This allows exclusion of many folders that fit a pattern without having to explicitly list ALL of them
-- Fix several bugs in the build for MAC
-- Finally fixed the requirement that you have to run MAC installation specifically from ~/Downloads/homer3_install. Added code to makesetup.pl to create a more intelligent setup.command which does not depend on execution from a specific folder.

* -- Fix build not working because need to copy ALL config files to installation folder
-- Improve launch performance by changing config file access to global variable

* -- Change setpath submodule download without git, to not ask questions about which branch to download. Instead derive the branch name from root folder name of parent app and default to master of it cannot be determined. This change somehow did not make it earlier into repo

* -- Update submodule references to latest development branch rev

* -- Change unit test to use global config variable rather than opening/closing ConfigFileClass objects
-- Update getVernum to match AV
-- Update submodule references

* DataTree submodule reference update and Mac support fixes (#89)

-- Update DataTree submodule reference to latest change containing fix for errors when there are no data files (because all have errors that prevent loading)

-- Fix more issues with install executable on MAC

* Gets info about latest version from GitHub API rather than openfnirs.org (#92)

* Gets info about latest version from GitHub API rather than openfnirs.org

* Version++

* -- Change setpaths to not download submodules by default. Use 'init' or 'update' options to download automatically. If using github desktop then github desktop download submodules automatically. (#95)

-- Change downloadSharedLibs to change .gitmodules origin url to point to where ever parent came from
-- Add .gitattributes to exclude files from branch merges such as submodule references and .gitmodules
-- Sync getVernum with Homre3. (getVernum will eventually be moved to Utils/Shared library)
-- Improve creation of installation file and installation process using myCopyFile.m

* -- Update submodule reference for development branch to latest
-- Fix .gitmodules to point to development branch instaead of master

* -- Fix setpaths gitSubmodulesUpdate not updating changes to submodules correctly

* -- Fix wrong path to DataTree to exclude from merges
-- Add .gitattributes in list of files to exclude fropm merges

* -- Update submodule references

* -- Update submodule reference to Utils

* Remove submodules: too hard to work with, not enough support from git

* Remove submodules: too hard to work with, not enough support from git

Co-authored-by: jayd1860 <[email protected]>
Co-authored-by: sreekanthkura7 <[email protected]>
Co-authored-by: Meryem Ayse Yucel <[email protected]>
Co-authored-by: Jay Dubb <[email protected]>
Co-authored-by: stephen scott tucker <[email protected]>
Co-authored-by: Jay Dubb <[email protected]>

* -- Step 2 of change to remove submodules references and replacing with hard copy of the DataTree and Utils libraries

Co-authored-by: jayd1860 <[email protected]>
Co-authored-by: sreekanthkura7 <[email protected]>
Co-authored-by: Meryem Ayse Yucel <[email protected]>
Co-authored-by: Jay Dubb <[email protected]>
Co-authored-by: stephen scott tucker <[email protected]>
Co-authored-by: Jay Dubb <[email protected]>

Co-authored-by: jayd1860 <[email protected]>
Co-authored-by: sreekanthkura7 <[email protected]>
Co-authored-by: Meryem Ayse Yucel <[email protected]>
Co-authored-by: Jay Dubb <[email protected]>
Co-authored-by: stephen scott tucker <[email protected]>
Co-authored-by: Jay Dubb <[email protected]>
  • Loading branch information
@jayd1860 @sreekanthkura7
@mayucel @sstucker
7 people authored Jan 20, 2022
1 parent 6ebf849 commit c83f79e
Show file tree
Hide file tree
Showing 158 changed files with 21,211 additions and 129 deletions.
1 change: 1 addition & 0 deletions DataTree/.numberfiles
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
67
255 changes: 255 additions & 0 deletions DataTree/AcquiredData/AcqDataClass.m
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,255 @@
classdef AcqDataClass < matlab.mixin.Copyable

properties (Access = private)
logger
end
properties (Access = protected)
errmsgs
end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% These methods must be implemented in any derived class
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
methods(Abstract)

% ---------------------------------------------------------
objnew = GetFormatVersion(obj, options);

% ---------------------------------------------------------
val = GetFormatVersionString(obj);

% ---------------------------------------------------------
t = GetTime(obj, iBlk)

% ---------------------------------------------------------
datamat = GetDataTimeSeries(obj, options, iBlk)

% ---------------------------------------------------------
SD = GetSDG(obj,option)

% ---------------------------------------------------------
srcpos = GetSrcPos(obj,option)

% ---------------------------------------------------------
detpos = GetDetPos(obj,option)

% ---------------------------------------------------------
ml = GetMeasList(obj, iBlk)


% ---------------------------------------------------------
wls = GetWls(obj)

% ---------------------------------------------------------
SetStims_MatInput(obj, s, t, CondNames)

% ---------------------------------------------------------
s = GetStims(obj, t)

% ---------------------------------------------------------
CondNames = GetConditions(obj)


% ---------------------------------------------------------
SetConditions(obj, CondNames)

% ---------------------------------------------------------
aux = GetAuxiliary(obj)

% ---------------------------------------------------------
SetStimDuration(obj, icond, duration);

% ---------------------------------------------------------
duration = GetStimDuration(obj, icond);

% ---------------------------------------------------------
n = GetDataBlocksNum(obj);

% ---------------------------------------------------------
[iDataBlks, ich] = GetDataBlocksIdxs(obj, ich);

% ---------------------------------------------------------
objnew = CopyMutable(obj, options);

end


methods

% -------------------------------------------------------
function Initialize(obj)
global logger
obj.logger = InitLogger(logger);
end


% -------------------------------------------------------
function err = Error(obj)
err = obj.GetError();
end


% ---------------------------------------------------------
function msg = GetErrorMsg(obj)
msg = '';
if isempty(obj)
msg = 'AcqDataClass object is empty';
return;
end
if isempty(obj.errmsgs)
return;
end
if ~obj.GetError()
return;
end
msg = obj.errmsgs{abs(obj.GetError())};
end


% -------------------------------------------------------
function FreeMemory(obj, filename)
if ~exist('filename','var')
filename = '';
end
if isempty(filename)
return;
end
obj.Initialize();
end


% ---------------------------------------------------------
function bbox = GetSdgBbox(obj)
bbox = [];

optpos = [obj.GetSrcPos('2D'); obj.GetDetPos('2D')];
if isempty(optpos)
return
end

xmax = max(optpos(:,1));
ymax = max(optpos(:,2));

xmin = min(optpos(:,1));
ymin = min(optpos(:,2));

width = xmax-xmin;
height = ymax-ymin;

if width==0
width = 1;
end
if height==0
height = 1;
end

px = width * 0.05;
py = height * 0.05;

bbox = [xmin-px, xmax+px, ymin-py, ymax+py];
end


% ----------------------------------------------------------------------------------
function varval = GetVar(obj, varname)
if ismethod(obj,['Get_', varname])
varval = eval( sprintf('obj.Get_%s()', varname) );
elseif ismethod(obj,['Get', varname])
varval = eval( sprintf('obj.Get%s()', varname) );
elseif isproperty(obj, varname)
varval = eval( sprintf('obj.%s', varname) );
else
varval = [];
end

% If varval is a class object with an IsEmpty() method and length of varval is one or
% less, then use it determine whether it's valid. If not, return empty as if it were
% really empty. This is useful when we want to copy only part of an object to save on space.
% For instance to reconstruct nirs style 's' variable from a SNIRF stim object we need the t
% property from a SNIRF data container, but we only need that one property not the whole thing
% so we initilize only the t property. We don't want this partially initialized data object
% to be found (by GetVar) and used directly in the proc stream processing, we only want to use
% indirectly to retrieve s from stim. So we do this check to see if it's valid. It won't be
% since it's only partially initialized.
if isa(varval, 'handle') && ismethod(varval,'IsEmpty')
if length(varval)==1 && varval(1).IsEmpty()
varval = [];
end
end
end


% ----------------------------------------------------------------------------------
function t = GetTimeCombined(obj)
% Function combines the time vectors for all data blocks into one time vectors.
t = obj.GetTime(1);
tStart = t(1);
tEnd = t(end);
tStep = mean(diff(t));

nBlks = obj.GetDataBlocksNum();
for iBlk=2:nBlks
t = obj.GetTime(iBlk);
if t(1) < tStart
tStart = t(1);
end
if t(end) > tEnd
tEnd = t(end);
end
if mean(diff(t)) < tStep
tStep = mean(diff(t));
end
end
t = tStart:tStep:tEnd;
end


% ----------------------------------------------------------------------------------
function data = GetStimData(~, ~)
data = [];
end


% ----------------------------------------------------------------------------------
function val = GetStimDataLabels(~, ~)
val = {};
end


% ----------------------------------------------------------------------------------
function b = equal(obj, obj2)
b = true;
if isempty(obj.GetFilename)
return;
end
if isempty(obj2.GetFilename)
return;
end
[~, fname1, ext1] = fileparts(obj.GetFilename);
[~, fname2, ext2] = fileparts(obj2.GetFilename);
if ~strcmpi([fname1, ext1], [fname2, ext2])
b = false;
end
end


% ----------------------------------------------------------------------------------
function status = Mismatch(obj, obj2)
status = 0;
msg = {};
if ~exist('obj2','var')
return;
end
if ~obj.equal(obj2)
[~, fname, ext] = fileparts(obj.GetFilename);
msg{1} = sprintf('WARNING: The acquisition file "%s" does not match the derived data in this group folder. ', [fname, ext]);
msg{2} = sprintf('Are you sure this acquisition file belongs in this group folder?');
obj.logger.Write([msg{:}])
end
end



end

end
Loading

0 comments on commit c83f79e

Please sign in to comment.