Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix magento_indexer_backlog_count_total #34

Merged
merged 2 commits into from
Jul 7, 2023
Merged

Fix magento_indexer_backlog_count_total #34

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c4feae7
change the way how magento_indexer_backlog_count_total calculates the…
vladyslav-sikailo-run-as-root Jul 6, 2023
b4c6e7e
add cleanup data patch to delete metrics with wrong structure
vladyslav-sikailo-run-as-root Jul 7, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 15 additions & 15 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,21 +48,21 @@ in the table and renders the correct response for prometheus.

The following metrics will be collected:

| Metric | Labels | TYPE | Help |
|:------------------------------------|:--------------------------------|:------|:-----------------------------------------------------------------------------|
| magento_orders_count_total | status, store_code | gauge | All Magento Orders |
| magento_orders_amount_total | status, store_code | gauge | Total amount of all Magento Orders |
| magento_order_items_count_total | status, store_code | gauge | Total count of orderitems |
| magento_cms_block_count_total | store_code | gauge | Total count of available cms blocks |
| magento_cms_page_count_total | store_code | gauge | Total count of available cms pages |
| magento_customer_count_total | store_code | gauge | Total count of available customer |
| magento_cronjob_broken_count_total | | gauge | Broken CronJobs occur when when status is pending but execution_time is set. |
| magento_cronjob_count_total | status, job_code | gauge | Total count of available CronJob Count. |
| magento_indexer_backlog_count_total | isValid, title, status | gauge | Total count of backlog item in indexer. |
| magento_shipments_count_total | source, store_code | counter| Count of Shipments created by store and source. |
| magento_catalog_category_count_total| status, menu_status, store_code | gauge | Count of Categories by store, status and menu status. |
| magento_store_count_total | status | gauge | Total count of Stores by status. |
| magento_website_count_total | | gauge | Total count websites. |
| Metric | Labels | TYPE | Help |
|:-------------------------------------|:--------------------------------|:--------|:--------------------------------------------------------------------------------|
| magento_orders_count_total | status, store_code | gauge | All Magento Orders |
| magento_orders_amount_total | status, store_code | gauge | Total amount of all Magento Orders |
| magento_order_items_count_total | status, store_code | gauge | Total count of orderitems |
| magento_cms_block_count_total | store_code | gauge | Total count of available cms blocks |
| magento_cms_page_count_total | store_code | gauge | Total count of available cms pages |
| magento_customer_count_total | store_code | gauge | Total count of available customer |
| magento_cronjob_broken_count_total | | gauge | Broken CronJobs occur when when status is pending but execution_time is set. |
| magento_cronjob_count_total | status, job_code | gauge | Total count of available CronJob Count. |
| magento_indexer_backlog_count_total | title | gauge | Total count of backlog item in indexer (the data from `indexer:status` command) |
| magento_shipments_count_total | source, store_code | counter | Count of Shipments created by store and source. |
| magento_catalog_category_count_total | status, menu_status, store_code | gauge | Count of Categories by store, status and menu status. |
| magento_store_count_total | status | gauge | Total count of Stores by status. |
| magento_website_count_total | | gauge | Total count websites. |

## Add you own Metric

Expand Down
268 changes: 51 additions & 217 deletions Test/Unit/Aggregator/Index/IndexerBacklogCountAggregatorTest.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,260 +4,94 @@

namespace RunAsRoot\PrometheusExporter\Test\Unit\Aggregator\Index;

use Magento\Framework\App\ResourceConnection;
use Magento\Framework\DB\Adapter\AdapterInterface;
use Magento\Framework\DB\Select;
use Magento\Framework\Indexer\IndexerInterface;
use Magento\Framework\Mview\View\ChangelogInterface;
use Magento\Framework\Mview\ViewInterface;
use Magento\Indexer\Model\Indexer\Collection;
use Magento\Indexer\Model\Indexer\CollectionFactory;
use Magento\Indexer\Model\Mview\View\State;
use PHPUnit\Framework\MockObject\MockObject;
use PHPUnit\Framework\TestCase;
use Psr\Log\LoggerInterface;
use RunAsRoot\PrometheusExporter\Aggregator\Index\IndexerBacklogCountAggregator;
use RunAsRoot\PrometheusExporter\Service\UpdateMetricService;
use RunAsRoot\PrometheusExporter\Service\UpdateMetricServiceInterface;

final class IndexerBacklogCountAggregatorTest extends TestCase
{
private const TABLE_CHANGELOG_1 = 'table_name_cl';
private const TABLE_CHANGELOG_2 = 'table_name_2_cl';
private const VIEW_ID_1 = 'some_name';
private const VIEW_ID_2 = 'some_name_another_name';
private const TABLE_MVIEW_STATE = 'mview_state';
private const METRIC_CODE = 'magento_indexer_backlog_count_total';

private IndexerBacklogCountAggregator $sut;

/** @var MockObject|UpdateMetricServiceInterface */
private $updateMetricService;

/** @var MockObject|CollectionFactory */
private $indexerCollectionFactory;

/** @var MockObject|LoggerInterface */
private $logger;

/** @var MockObject|Collection */
private $indexerCollection;

/** @var MockObject|ResourceConnection */
private $resourceConnection;
private MockObject|UpdateMetricServiceInterface $updateMetricService;
private MockObject|Collection $indexerCollection;

protected function setUp(): void
{
$this->updateMetricService = $this->createMock(UpdateMetricService::class);
$this->indexerCollectionFactory = $this->createMock(CollectionFactory::class);
$indexerCollectionFactory = $this->createMock(CollectionFactory::class);
$this->indexerCollection = $this->createMock(Collection::class);
$this->logger = $this->createMock(LoggerInterface::class);
$this->resourceConnection = $this->createMock(ResourceConnection::class);

$this->indexerCollectionFactory->method('create')->willReturn($this->indexerCollection);
$indexerCollectionFactory->method('create')->willReturn($this->indexerCollection);

$this->sut = new IndexerBacklogCountAggregator(
$this->updateMetricService,
$this->indexerCollectionFactory,
$this->resourceConnection,
$this->logger
);
$this->sut = new IndexerBacklogCountAggregator($this->updateMetricService, $indexerCollectionFactory);
}

public function testAggregate(): void
public function testItAggregatesIndexersChangelogCount(): void
{
$this->setUpIndexCollection();
$this->setUpSelects();

$lables1 = [
'isValid' => true,
'title' => 'name',
'status' => 'success',
];

$lables2 = [
'isValid' => true,
'title' => 'other_name',
'status' => 'failed',
];
$indexer1 = $this->createMock(IndexerInterface::class);
$indexer2 = $this->createMock(IndexerInterface::class);
$view1 = $this->createMock(ViewInterface::class);
$view2 = $this->createMock(ViewInterface::class);
$changelog1 = $this->createMock(ChangelogInterface::class);
$changelog2 = $this->createMock(ChangelogInterface::class);
$stateMock1 = $this->createMock(State::class);
$stateMock2 = $this->createMock(State::class);

$testCurrentVersionId1 = '1111';
$testCurrentVersionId2 = '2222';
$testStateVersion1 = '1100';
$testStateVersion2 = '2200';
$testPendingCount1 = 11;
$testPendingCount2 = 22;

$indexer1->method('getTitle')->willReturn(self::VIEW_ID_1);
$indexer2->method('getTitle')->willReturn(self::VIEW_ID_2);

$indexer1->expects($this->once())->method('getView')->willReturn($view1);
$indexer2->expects($this->once())->method('getView')->willReturn($view2);
$view1->expects($this->once())->method('getChangelog')->willReturn($changelog1);
$view2->expects($this->once())->method('getChangelog')->willReturn($changelog2);
$view1->expects($this->once())->method('getState')->willReturn($stateMock1);
$view2->expects($this->once())->method('getState')->willReturn($stateMock2);

$changelog1->method('getVersion')->willReturn($testCurrentVersionId1);
$changelog2->method('getVersion')->willReturn($testCurrentVersionId2);
$stateMock1->method('getVersionId')->willReturn($testStateVersion1);
$stateMock2->method('getVersionId')->willReturn($testStateVersion2);

$changelog1->method('getList')
->with($testStateVersion1, $testCurrentVersionId1)
->willReturn(range(1, $testPendingCount1));
$changelog2->method('getList')
->with($testStateVersion2, $testCurrentVersionId2)
->willReturn(range(1, $testPendingCount2));

$this->indexerCollection->expects($this->once())->method('getItems')->willReturn([ $indexer1, $indexer2 ]);

$lables1 = [ 'title' => self::VIEW_ID_1 ];
$lables2 = [ 'title' => self::VIEW_ID_2 ];

$this->updateMetricService
->expects($this->at(0))
->method('update')
->with(self::METRIC_CODE, '320', $lables1);
->with(self::METRIC_CODE, '11', $lables1);

$this->updateMetricService
->expects($this->at(1))
->method('update')
->with(self::METRIC_CODE, '33', $lables2);
->with(self::METRIC_CODE, '22', $lables2);

$this->sut->aggregate();
}

public function testExceptionBehaviour(): void
{
$this->setUpIndexCollection();
$this->setUpSelects(true);

$this->logger
->expects($this->at(0))
->method('critical')
->with(...['ERROR CTNEE']);

$this->logger
->expects($this->at(1))
->method('critical')
->with(...['ERROR RE']);

$this->updateMetricService
->expects($this->never())
->method('update');

$this->sut->aggregate();
}

private function setUpSelects(bool $throwException = false): void
{
$select1 = $this->createMock(Select::class);
$select2 = $this->createMock(Select::class);
$select3 = $this->createMock(Select::class);
$select4 = $this->createMock(Select::class);

$connection = $this->createMock(AdapterInterface::class);
$this->resourceConnection->expects($this->once())->method('getConnection')
->willReturn($connection);
$connection->expects($this->exactly(4))
->method('select')
->will($this->onConsecutiveCalls($select1, $select2, $select3, $select4));
$connection->expects($this->exactly(4))
->method('getTableName')
->willReturnMap(
[
[self::TABLE_CHANGELOG_1, self::TABLE_CHANGELOG_1],
[self::TABLE_CHANGELOG_2, self::TABLE_CHANGELOG_2],
[self::TABLE_MVIEW_STATE, self::TABLE_MVIEW_STATE]
]
);
$select1 = $this->setUpSelectChangelog($select1, self::TABLE_CHANGELOG_1);
$select2 = $this->setUpSelectMviewState($select2, self::VIEW_ID_1);
$select3 = $this->setUpSelectChangelog($select3, self::TABLE_CHANGELOG_2);
$select4 = $this->setUpSelectMviewState($select4, self::VIEW_ID_2);

if (!$throwException) {
$connection->expects($this->exactly(4))
->method('fetchOne')
->willReturnMap(
[
[$select1, [], 550],
[$select2, [], 230],
[$select3, [], 99],
[$select4, [], 66]
]
);
} else {
$connection->expects($this->exactly(4))
->method('fetchOne')
->will(
$this->returnCallback(function ($arg) use ($select2, $select4) {
if (spl_object_id($select2) === spl_object_id($arg)) {
throw new \Zend_Db_Exception('ERROR CTNEE');
} elseif (spl_object_id($select4) === spl_object_id($arg)) {
throw new \Zend_Db_Exception('ERROR RE');
} else {
return 20;
}
})
);
}
}

private function setUpSelectMviewState(Select $select, string $viewId): Select
{
$select->expects($this->once())->method('from')->with(self::TABLE_MVIEW_STATE)
->willReturn($select);
$select->expects($this->once())->method('reset')->with(Select::COLUMNS)
->willReturn($select);
$select->expects($this->once())->method('where')->with('view_id = ?', $viewId)
->willReturn($select);
$select->expects($this->once())->method('columns')->with(['version_id'])
->willReturn($select);

return $select;
}

private function setUpSelectChangelog(Select $select, string $changelogTableName): Select
{
$select->expects($this->once())->method('from')->with($changelogTableName)
->willReturn($select);
$select->expects($this->once())->method('reset')->with(Select::COLUMNS)
->willReturn($select);
$select->expects($this->once())->method('order')->with('version_id DESC')
->willReturn($select);
$select->expects($this->once())->method('columns')->with(['version_id'])
->willReturn($select);

return $select;
}

private function setUpIndexCollection(): void
{
$indexer1 = $this->createMock(IndexerInterface::class);
$indexer2 = $this->createMock(IndexerInterface::class);
$view1 = $this->createMock(ViewInterface::class);
$view2 = $this->createMock(ViewInterface::class);
$changelog1 = $this->createMock(ChangelogInterface::class);
$changelog2 = $this->createMock(ChangelogInterface::class);

$indexer1->method('isValid')->willReturn(true);
$indexer1->method('getTitle')->willReturn('name');
$indexer1->method('getStatus')->willReturn('success');

$indexer2->method('isValid')->willReturn(true);
$indexer2->method('getTitle')->willReturn('other_name');
$indexer2->method('getStatus')->willReturn('failed');

$indexer1
->expects($this->once())
->method('getView')
->willReturn($view1);

$indexer2
->expects($this->once())
->method('getView')
->willReturn($view2);

$view1
->expects($this->once())
->method('getChangelog')
->willReturn($changelog1);

$view2
->expects($this->once())
->method('getChangelog')
->willReturn($changelog2);

$view1
->expects($this->once())
->method('getId')
->willReturn(self::VIEW_ID_1);

$view2
->expects($this->once())
->method('getId')
->willReturn(self::VIEW_ID_2);

$changelog1
->expects($this->once())
->method('getName')
->willReturn(self::TABLE_CHANGELOG_1);

$changelog2
->expects($this->once())
->method('getName')
->willReturn(self::TABLE_CHANGELOG_2);

$this->indexerCollection
->expects($this->once())
->method('getItems')
->willReturn([$indexer1, $indexer2]);
}
}
Loading