Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Master merge ursa 21.10.0 #13151

Merged
merged 68 commits into from
Feb 24, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
926b999
PLAT-25122-ffmpeg-audio-mapping-fix
anatolkaltura Feb 2, 2025
6e9424e
VCP-19989: add S3 DF assume role support for customer or Kaltura ARN
amiras89 Feb 2, 2025
67e8d96
verify required classes exists + tweaks
amiras89 Feb 2, 2025
43e0bef
do not override $this->s3Region until needed
amiras89 Feb 2, 2025
b5d24eb
simplify + fix static classname
amiras89 Feb 2, 2025
1e1a451
if region provided, take it
amiras89 Feb 3, 2025
5fead82
refactor flow
amiras89 Feb 3, 2025
c3aa829
remove
amiras89 Feb 3, 2025
c124cbd
remove
amiras89 Feb 3, 2025
c98f255
remove description for s3 df configure
amiras89 Feb 3, 2025
d5b27d6
SUP-46309 Close clip concat jobs in KClipConcatCloser if all child jo…
Feb 3, 2025
92777bc
Update BatchService.php
bw-kaltura Feb 3, 2025
2934735
keep s3 DF s3Region even if using cloud_storage s3Arn
amiras89 Feb 4, 2025
9d6a6ad
add description to s3 drop folder S3 Settings
amiras89 Feb 4, 2025
e043ecf
Refactor resources names retrieval code when generating iCal files
danbar0kal Feb 4, 2025
db06382
rename S3IAMRole to s3Arn since S3DropFolderFile send $s3Options whic…
amiras89 Feb 4, 2025
a4c5cfe
SUP-44096 Use KS user name in email report instead of admin name
Feb 4, 2025
c9d637a
Refactor resources names retrieval code when generating iCal files
danbar0kal Feb 4, 2025
79572c5
Refactor resources names retrieval code when generating iCal files
danbar0kal Feb 5, 2025
b2fad94
fix papi feedback
amiras89 Feb 5, 2025
74221e7
change logic get s3Arn from drop folder worker config
amiras89 Feb 5, 2025
aead7f8
change s3Arn to useS3Arn
amiras89 Feb 5, 2025
212132b
add validation
amiras89 Feb 6, 2025
2f2d831
modify df description
amiras89 Feb 6, 2025
709bf51
fix: shouldUseClosedCaptions func
ImryAshu Feb 9, 2025
c5288d0
set "remember" to new DF checkbox to show db value
amiras89 Feb 9, 2025
d2c6830
set "tag => div" over "dt" since there is no need for a list
amiras89 Feb 9, 2025
7c5115f
Updated version - Ursa-21.10.0
drorsou Feb 10, 2025
7c9fea0
Merge pull request #13131 from kaltura/Ursa-21.10.0-update-version
drorsou Feb 10, 2025
cd4fe43
changed to 'Until entry is ready'
danbar0kal Feb 10, 2025
0a1032b
Merge pull request #13132 from kaltura/Ursa-21.10.0-PLAT-25144
danbar0kal Feb 10, 2025
5c8cf69
SUP-46309 add ABORTED status
Feb 10, 2025
e1a0bdd
SUP-46309 update release notes
Feb 10, 2025
b52801c
SUP-44096 update
Feb 10, 2025
1db6548
SUP-44099 Handle single quotes in metadata to prevent failure in tran…
Feb 10, 2025
0a5af42
Update kXsd.php
bw-kaltura Feb 10, 2025
9dae39d
Merge pull request #13119 from kaltura/Ursa-21.9.0-SUP-44096
bw-kaltura Feb 10, 2025
2beedbe
SUP-44099 add
Feb 10, 2025
e9a7ba2
Merge pull request #13133 from kaltura/Ursa-21.10.0-SUP-44099
bw-kaltura Feb 10, 2025
9c60fa2
Merge pull request #13116 from kaltura/Ursa-21.9.0-SUP-46309
bw-kaltura Feb 10, 2025
11e99f0
Merge pull request #13117 from kaltura/Ursa-21.9.0-PLAT-25129
danbar0kal Feb 11, 2025
66655f1
Add missing dependencies - Transcript,ScheduledEventNotification
hilak Feb 11, 2025
97b9402
Merge pull request #13134 from kaltura/Ursa-21.9.0-PLAT-25122-ffmpeg-…
anatolkaltura Feb 11, 2025
33e4f5d
add full_name header to vpaas user usage report
Feb 11, 2025
4bd7449
Merge pull request #13136 from kaltura/fix-should-use-cc
yossipapi Feb 11, 2025
66d2e1c
Merge pull request #13135 from kaltura/Ursa-21.10.0-AN-23459
olampert Feb 11, 2025
8ac2386
Update release notes
hilak Feb 11, 2025
3183471
Add EXISTS criterion for user ExternalID field in esearch
hilak Feb 11, 2025
8d4e019
Update release-notes.md
hilak Feb 11, 2025
d3c6849
Merge pull request #13137 from kaltura/Ursa-21.10.0-addReachPluginDeps
hilak Feb 12, 2025
bc26946
Merge pull request #13138 from kaltura/Ursa-21.10.0-addExternalIDExis…
hilak Feb 12, 2025
dbe8fa6
use new drop_folder map to fetch s3Arn rather then taking from worker…
amiras89 Feb 16, 2025
d86517c
PLAT-25025: New RSVP UserEntry Type
drorsou Feb 16, 2025
19abdc2
SUP-46716 Increase limit for kuser about_me to 8192
Feb 16, 2025
a55b17e
Merge pull request #13141 from kaltura/Ursa-21.10.0-SUP-46716
bw-kaltura Feb 17, 2025
7250cf6
load s3Arn in runtime from runtime_config
amiras89 Feb 17, 2025
ff85c8a
fix admin add df description
amiras89 Feb 17, 2025
b33ac82
NOPLAT-Fix-Handling-Ultra-High-FPS-sources
anatolkaltura Feb 17, 2025
e248ba0
Merge pull request #13144 from kaltura/anatolkaltura-patch-386470
anatolkaltura Feb 18, 2025
49839ce
NPI-166:add genie admin
NadavHarnik Feb 18, 2025
90f3600
Merge pull request #13145 from kaltura/Ursa-21.10.0-NPI-166
NadavHarnik Feb 18, 2025
1e607a6
Merge pull request #13142 from kaltura/Ursa-21.10.0-PLAT-25025
drorsou Feb 18, 2025
9a6e4ed
throw Exception and not kFileTransferMgrException to show error msg i…
amiras89 Feb 19, 2025
f30bd50
add insert validation when adding df with useS3Arn "Bucket Policy All…
amiras89 Feb 19, 2025
c210df0
fix test typo
amiras89 Feb 19, 2025
8222c8a
change exception error
amiras89 Feb 19, 2025
f0f4706
change test intentinal typo
amiras89 Feb 19, 2025
3b48439
Merge pull request #13112 from kaltura/Ursa-21.9.0-VCP-19989
amiras89 Feb 19, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion VERSION.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Ursa-21.9.0
Ursa-21.10.0
17 changes: 10 additions & 7 deletions alpha/apps/kaltura/lib/batch2/kFlowHelper.php
Original file line number Diff line number Diff line change
Expand Up @@ -3330,12 +3330,12 @@ public static function handleReportExportFinished(BatchJob $dbBatchJob, kReportE
$expiryInDays = $expiry / 60 / 60 / 24;
if ($expiryInDays >= 1)
{
$params = array($dbBatchJob->getPartner()->getName(), $time, $dbBatchJob->getId(), implode('<BR>', $links), $expiryInDays, self::DAYS, $validUntil);
$params = array($data->getRecipientName(), $time, $dbBatchJob->getId(), implode('<BR>', $links), $expiryInDays, self::DAYS, $validUntil);
}
else
{
$expiryInHours = $expiry / 60 / 60;
$params = array($dbBatchJob->getPartner()->getName(), $time, $dbBatchJob->getId(), implode('<BR>', $links), $expiryInHours, self::HOURS, $validUntil);
$params = array($data->getRecipientName(), $time, $dbBatchJob->getId(), implode('<BR>', $links), $expiryInHours, self::HOURS, $validUntil);
}
$titleParams = array($data->getReportsGroup(), $time);

Expand All @@ -3349,7 +3349,8 @@ public static function handleReportExportFinished(BatchJob $dbBatchJob, kReportE
kConf::get("report_sender_name"),
$data->getRecipientEmail(),
$params,
$titleParams
$titleParams,
$data->getRecipientName()
);
return $dbBatchJob;
}
Expand All @@ -3358,7 +3359,7 @@ public static function handleReportExportFailed(BatchJob $dbBatchJob, kReportExp
{
$time = date("m-d-y H:i", $data->getTimeReference() + $data->getTimeZoneOffset());
$email_id = MailType::MAIL_TYPE_REPORT_EXPORT_FAILURE;
$params = array($dbBatchJob->getPartner()->getName(), $time, $dbBatchJob->getId(),
$params = array($data->getRecipientName(), $time, $dbBatchJob->getId(),
$dbBatchJob->getErrType(), $dbBatchJob->getErrNumber());
$titleParams = array($data->getReportsGroup(), $time);

Expand All @@ -3372,7 +3373,8 @@ public static function handleReportExportFailed(BatchJob $dbBatchJob, kReportExp
kConf::get("report_sender_name"),
$data->getRecipientEmail(),
$params,
$titleParams
$titleParams,
$data->getRecipientName()
);
return $dbBatchJob;
}
Expand All @@ -3381,7 +3383,7 @@ public static function handleReportExportAborted(BatchJob $dbBatchJob, kReportEx
{
$time = date("m-d-y H:i", $data->getTimeReference() + $data->getTimeZoneOffset());
$email_id = MailType::MAIL_TYPE_REPORT_EXPORT_ABORT;
$params = array($dbBatchJob->getPartner()->getName(), $time, $dbBatchJob->getId());
$params = array($data->getRecipientName(), $time, $dbBatchJob->getId());
$titleParams = array($data->getReportsGroup(), $time);

kJobsManager::addMailJob(
Expand All @@ -3394,7 +3396,8 @@ public static function handleReportExportAborted(BatchJob $dbBatchJob, kReportEx
kConf::get("report_sender_name"),
$data->getRecipientEmail(),
$params,
$titleParams
$titleParams,
$data->getRecipientName()
);
return $dbBatchJob;
}
Expand Down
1 change: 1 addition & 0 deletions alpha/apps/kaltura/lib/batch2/kJobsManager.php
Original file line number Diff line number Diff line change
Expand Up @@ -1980,6 +1980,7 @@ public static function addExportReportJob(KalturaReportExportParams $params)
$coreParams = $params->toObject();

$jobData->setRecipientEmail($coreParams->getRecipientEmail());
$jobData->setRecipientName($coreParams->getRecipientName());
$jobData->setReportItems($coreParams->getReportItems());

$offset = $timeOffsetSeconds - ($params->timeZoneOffset * 60);// Convert minutes to seconds
Expand Down
21 changes: 21 additions & 0 deletions alpha/apps/kaltura/lib/batch2/model/kReportExportJobData.php
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,11 @@ class kReportExportJobData extends kJobData
*/
protected $recipientEmail;

/**
* @var string
*/
protected $recipientName;

/**
* @var array
*/
Expand Down Expand Up @@ -62,6 +67,22 @@ public function setRecipientEmail($recipientEmail)
$this->recipientEmail = $recipientEmail;
}

/**
* @return string
*/
public function getRecipientName()
{
return $this->recipientName;
}

/**
* @param string $recipientName
*/
public function setRecipientName($recipientName)
{
$this->recipientName = $recipientName;
}

/**
* @return array
*/
Expand Down
10 changes: 3 additions & 7 deletions alpha/apps/kaltura/lib/kManifestRenderers.php
Original file line number Diff line number Diff line change
Expand Up @@ -1031,15 +1031,11 @@ protected function getManifestFlavors()

protected function shouldUseClosedCaptions(): bool
{
$dbEntry = entryPeer::retrieveByPK($this->entryId);
if($dbEntry)
foreach ($this->contributors as $contributor)
{
foreach ($this->contributors as $contributor)
if($contributor instanceof WebVttCaptionsManifestEditor)
{
if($contributor instanceof WebVttCaptionsManifestEditor && $dbEntry->getType() == entryType::LIVE_STREAM)
{
return false;
}
return false;
}
}
return true;
Expand Down
5 changes: 3 additions & 2 deletions alpha/apps/kaltura/lib/reports/kKavaVpaasReports.php
Original file line number Diff line number Diff line change
Expand Up @@ -47,13 +47,14 @@ class kKavaVpaasReports extends kKavaReports
self::REPORT_DIMENSION_MAP => array(
'kuser_id' => self::DIMENSION_KUSER_ID,
'name' => self::DIMENSION_KUSER_ID,
'full_name' => self::DIMENSION_KUSER_ID,
'partner_id' => self::DIMENSION_KUSER_ID,
),
self::REPORT_ENRICH_DEF => array(
self::REPORT_ENRICH_OUTPUT => array('name', 'partner_id'),
self::REPORT_ENRICH_OUTPUT => array('name', 'full_name', 'partner_id'),
self::REPORT_ENRICH_FUNC => 'kKavaVpaasReports::getUsersInfoVpaas',
self::REPORT_ENRICH_CONTEXT => array(
'columns' => array('PUSER_ID', 'PARTNER_ID'),
'columns' => array('PUSER_ID', 'TRIM(CONCAT(FIRST_NAME, " ", LAST_NAME))', 'PARTNER_ID'),
)
),
),
Expand Down
21 changes: 21 additions & 0 deletions alpha/apps/kaltura/lib/reports/kReportExportParams.php
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,11 @@ class kReportExportParams
*/
protected $recipientEmail;

/**
* @var string
*/
protected $recipientName;

/**
* @var array
*/
Expand Down Expand Up @@ -38,6 +43,22 @@ public function setRecipientEmail($recipientEmail)
$this->recipientEmail = $recipientEmail;
}

/**
* @return string
*/
public function getRecipientName()
{
return $this->recipientName;
}

/**
* @param string $recipientName
*/
public function setRecipientName($recipientName)
{
$this->recipientName = $recipientName;
}

/**
* @return array
*/
Expand Down
13 changes: 13 additions & 0 deletions alpha/lib/model/kuser.php
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,8 @@ public function __construct()

const MAX_NAME_LEN = 40;

const MAX_ABOUT_ME_LEN = 4096;

const MAX_COUNTRY_LEN = 2;

private $roughcut_count = -1;
Expand Down Expand Up @@ -1686,4 +1688,15 @@ public function getCapabilities()
$capabilities = $this->getFromCustomData(self::CAPABILITIES, null, '');
return ($capabilities == '') ? array() : explode(',', $capabilities);
}

public function setAboutMe($v)
{
PeerUtils::setExtension($this, $v, self::MAX_ABOUT_ME_LEN, __FUNCTION__);
return parent::setAboutMe(kString::alignUtf8String($v, self::MAX_ABOUT_ME_LEN));
}

public function getAboutMe()
{
return parent::getAboutMe() . PeerUtils::getExtension($this, __FUNCTION__);
}
}
1 change: 1 addition & 0 deletions api_v3/lib/types/KalturaBaseUser.php
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ class KalturaBaseUser extends KalturaObject implements IRelatedFilterable

/**
* @var string
* @maxLength 8192
*/
public $description;

Expand Down
5 changes: 5 additions & 0 deletions api_v3/lib/types/batch/KalturaReportExportJobData.php
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,11 @@ class KalturaReportExportJobData extends KalturaJobData
*/
public $recipientEmail;

/**
* @var string
*/
public $recipientName;

/**
* @var KalturaReportExportItemArray
*/
Expand Down
6 changes: 6 additions & 0 deletions api_v3/lib/types/report/export/KalturaReportExportParams.php
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,11 @@ class KalturaReportExportParams extends KalturaObject
*/
public $recipientEmail;

/**
* @var string
*/
public $recipientName;

/**
* Time zone offset in minutes (between client to UTC)
* @var int
Expand All @@ -35,6 +40,7 @@ class KalturaReportExportParams extends KalturaObject
private static $map_between_objects = array
(
"recipientEmail",
"recipientName",
"reportItems",
"reportsItemsGroup",
"baseUrl"
Expand Down
14 changes: 14 additions & 0 deletions api_v3/services/BatchService.php
Original file line number Diff line number Diff line change
Expand Up @@ -694,4 +694,18 @@ public function deRegisterSchedulerAction($host)
SchedulerPeer::deleteBySchedulerConfigId($scheduler->getConfiguredId());
return true;
}

/**
* batch getAllChildJobs action returns all child jobs
*
* @action getAllChildJobs
* @param int $rootJobId root job id
* @return KalturaBatchJobArray
*/
function getAllChildJobsAction($rootJobId)
{
$rootJob = BatchJobPeer::retrieveByPK($rootJobId);
$result = KalturaBatchJobArray::fromBatchJobArray($rootJob->getChildJobs());
return $result;
}
}
19 changes: 7 additions & 12 deletions api_v3/services/ReportService.php
Original file line number Diff line number Diff line change
Expand Up @@ -440,20 +440,15 @@ public function getCsvFromStringParamsAction($id, $params = null, $excludedField
public function exportToCsvAction(KalturaReportExportParams $params)
{
$this->validateReportExportParams($params);

$kuser = kCurrentContext::getCurrentKsKuser();
$partner = PartnerPeer::retrieveByPK(kCurrentContext::getCurrentPartnerId());
if (!$params->recipientEmail)
{
$kuser = kCurrentContext::getCurrentKsKuser();
if ($kuser)
{
$params->recipientEmail = $kuser->getEmail();
}
else
{
$partnerId = kCurrentContext::getCurrentPartnerId();
$partner = PartnerPeer::retrieveByPK($partnerId);
$params->recipientEmail = $partner->getAdminEmail();
}
$params->recipientEmail = $kuser ? $kuser->getEmail() : $partner->getAdminEmail();
}
if (!$params->recipientName)
{
$params->recipientName = $kuser ? $kuser->getFullName() : $partner->getName();
}

if (!is_null($params->reportsItemsGroup) && strlen($params->reportsItemsGroup) > self::MAX_EXPORT_GROUP_NAME_LENGTH)
Expand Down
11 changes: 10 additions & 1 deletion batch/batches/ClipConcat/KClipConcatCloser.class.php
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,16 @@ public static function getType()
*/
protected function exec(KalturaBatchJob $job)
{
return $this->checkTimeout($job);
$children = KBatchBase::$kClient->batch->getAllChildJobs($job->id);
$doneStatuses = array(KalturaBatchJobStatus::FINISHED, KalturaBatchJobStatus::FAILED, KalturaBatchJobStatus::ABORTED);
foreach ($children as $child)
{
if (!in_array($child->status, $doneStatuses))
{
return $this->checkTimeout($job);
}
}
return $this->closeJob($job, null, null, null, KalturaBatchJobStatus::FINISHED);
}

private function checkTimeout(KalturaBatchJob $job)
Expand Down
2 changes: 2 additions & 0 deletions configurations/plugins.template.ini
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,7 @@ CaptionAssetEventNotifications
AttachmentAssetEventNotifications
TranscriptAssetEventNotifications
DropFolderEventNotifications
ScheduleEventNotifications
Kontiki
Velocix
Wowza
Expand Down Expand Up @@ -146,3 +147,4 @@ Interactivity
;Room
VirtualEvent
;SessionCuePoint
Rsvp
8 changes: 8 additions & 0 deletions deployment/permissions/service.batch.ini
Original file line number Diff line number Diff line change
Expand Up @@ -193,3 +193,11 @@ permissionItem87.param5 =
permissionItem87.tags =
permissionItem87.permissions = BATCH_BASE

permissionItem88.service = batch
permissionItem88.action = getAllChildJobs
permissionItem88.partnerId = -1
permissionItem88.param3 =
permissionItem88.param4 =
permissionItem88.param5 =
permissionItem88.tags =
permissionItem88.permissions = -1>BATCH_BASE
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
<?php
/**
* @package deployment
* @subpackage ursa.roles_and_permissions
*/

$script = realpath(dirname(__FILE__) . '/../../../../') . '/alpha/scripts/utils/permissions/addPermissionsAndItems.php';
$config = realpath(dirname(__FILE__)) . '/../../../permissions/service.batch.ini';
passthru("php $script $config");
26 changes: 15 additions & 11 deletions infra/cdl/kdl/KDLOperatorFfmpeg6_0.php
Original file line number Diff line number Diff line change
Expand Up @@ -36,28 +36,31 @@ public function generateSinglePassCommandLine(KDLFlavor $design, KDLFlavor $targ
$keyVidFilters = $flGrKey+1;
}
}

if(isset($target->_forGenericSource) && $target->_forGenericSource==true) {
$target->multyAudioMapping = null;
$generalAudMapping = 'a';
// Audio mapping handling is needed only if there is an audio stream to generate ...
if(isset($target->_audio)) {
if(isset($target->_forGenericSource) && $target->_forGenericSource==true) {
$target->multyAudioMapping = null;
$generalAudMapping = 'a';
}
else $generalAudMapping = 'a:0';
}
else $generalAudMapping = 'a:0';

$mappingStr = null;
/*
* Following code adapts the FFM4.4 and earlier) cmdLines to FFM6 standard.
* Following code adapts the FFM4.4 (and earlier) cmdLines to FFM6 standard.
* FFM6 'resolves' the ambiguity in previous versions while using both
* the filter_complex and stream mappings.
* Starting with FFM6 the streams that are used by the filter_complex are
* 'mapped' automatically. When in this case explicit '-map' is used as well,
* it is ADDED to the filter_complex 'mapped' streams, resulting converted
* streams duplication. FFM4 and older versions don't duplicate the
* transcodided streams.
* The solution is to use 'named' filter graphs out connections and explicitly
* to map them, and to avoid mapping of the source streams.
* The solution is -
* - using 'named' filter graphs out connections and explicitly mapping them
* - avoiding 'redundant' source stream mapping
*
* Another task is to force the output vid stream as the 1st stream,
* and the aud as the 2nd. It is needed for the multy-entry-edit feature.
* Another task is to force the output vid stream to be the 1st stream,
* and the aud to be the 2nd. It is needed for the multy-entry-edit feature.
*
* !!! IMPORTANT !!!
* This syntax is fully compatibale with the FFM4.4 as well.
Expand All @@ -81,7 +84,8 @@ public function generateSinglePassCommandLine(KDLFlavor $design, KDLFlavor $targ
$mappingStr = '-map \'[vout]\' ';
if(isset($target->multyAudioMapping))
$mappingStr.= implode(' ', $target->multyAudioMapping);
else $mappingStr.= "-map $generalAudMapping";
else if(isset($generalAudMapping))
$mappingStr.= "-map $generalAudMapping";
}
// only aud filters are used ==>
// add aout connection to the audio filter graph.
Expand Down
Loading