Scanning search instead of individual links.

[joeish]

Scan entire search instead of individual links, is this possible?

[AlexP11223]

See linksBuilder (in the code, does not seem to be documented).

[joeish]

Do you know what would need to be changed in order to scan multiple items in search and display output instead of each product url?

[AlexP11223]

See examples in https://github.com/jef/streetmerchant/search?q=linksBuilder

[joeish]

import {Store} from './store';
import {getProductLinksBuilder} from './helpers/card';
export const CanadaComputers: Store = {
currency: '$',
labels: {
inStock: {
container: 'div[class="pi-prod-availability"]',
text: ['Online In Stock'],
},
maxPrice: {
container: '.h2-big > strong:nth-child(1)',
euroFormat: false,
},
},
links: [
{
brand: 'test:brand',
model: 'test:model',
series: 'test:series',
url:
'https://www.canadacomputers.com/product_info.php?item_id=167320&cPath=27_1046_365&language=en',
},
],
linksBuilder: {
builder: getProductLinksBuilder({
productsSelector: '.addCartSearch',
sitePrefix: 'https://www.canadacomputers.com',
titleSelector: '.text-center',
urlSelector: 'a[href]',
}),
urls: [

  {
    series: '3070',
    url:
      'https://www.canadacomputers.com/search/results_details.php?language=en&keywords=3070&cpath=43;557',
  },
  {
    series: '3090',
    url:
      'https://www.canadacomputers.com/search/results_details.php?language=en&keywords=3090&cpath=43;557',
  },
],

},
name: 'canadacomputers',
waitUntil: 'domcontentloaded',
};

Stuck running links builder, any ideas?

[AlexP11223]

It does not open for me.

Also format the code https://docs.github.com/en/github/writing-on-github/basic-writing-and-formatting-syntax#quoting-code

See how link builder is implemented, maybe you are using selectors wrong.

streetmerchant/src/store/model/helpers/card.ts

Lines 19 to 67 in ba35706

	export function getProductLinksBuilder(options: LinksBuilderOptions) {
	return (docElement: cheerio.Cheerio, series: Series): Link[] => {
	const productElements = docElement.find(options.productsSelector);
	const links: Link[] = [];
	for (let i = 0; i < productElements.length; i++) {
	const productElement = productElements.eq(i);
	const titleElement = productElement.find(options.titleSelector).first();
	const title = options.titleAttribute
	? titleElement.attr()?.[options.titleAttribute]
	: titleElement.text()?.replace(/\n/g, ' ').trim();
	if (!title) {
	continue;
	}
	let urlElement = titleElement;
	if (options.urlSelector) {
	urlElement = urlElement.find(options.urlSelector).first();
	}
	let url = urlElement.attr()?.href;
	if (!url) {
	continue;
	}
	if (isPartialUrlRegExp.exec(url)) {
	url = options.sitePrefix + url;
	}
	const card = parseCard(title);
	if (card) {
	links.push({
	brand: card.brand as any,
	model: card.model,
	series,
	url,
	});
	} else {
	logger.error(`Failed to parse card: ${title}`, {url});
	}
	}
	return links;
	};
	}

Also you can add logging there to make debugging easier, such as logger.info(title) on line 30.

[AlexP11223]

Also if the products in search are loaded via ajax, this may help: #2437

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 10 days

[github-actions]

This issue has been closed because it is stale. Reopen if necessary.