Fixed the crawler behavior in all three versions
Should now behave as intended without skipping directories
The problem was with how the link handling was done.
Now there is proper absolute URL handling
Fixed the crawler behavior in all three versions
Should now behave as intended without skipping directories
The problem was with how the link handling was done.
Now there is proper absolute URL handling