Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Community Marketplace Add-on Service - initial contribution #2405

Merged
merged 14 commits into from
Sep 14, 2021

Conversation

ghys
Copy link
Member

@ghys ghys commented Jun 15, 2021

This is an initial attempt at reimplementing an add-on service as an alternative to the distribution (KarafAddonService), similar to the (sooner or later) discontinued Eclipse IoT Marketplace from OH2.
It is inspired and implements discussions in #2070, #2058, openhab/openhab-addons#8448
and borrows code and concepts from:

The community marketplace sources its content from a special category on Discourse (https://community.openhab.org/c/marketplace/69) which is still hidden to the general public in order to avoid confusion (but if you're in the staff group you may have seen it already). If you intend to review this PR and don't happen to have access to https://community.openhab.org/c/marketplace/69, I can give you access and provide you with an API key to put in Settings > Community Marketplace, ask on Discourse via PM.

The add-on service is able to list and import these types of add-ons:

  • bindings
    • direct link to a .jar file

2021-06-15_10-37-16

  • rule templates
    • direct link to a .json or .yaml file
    • code block in JSON or YAML format

2021-06-15_10-37-33

  • UI widgets
    • direct link to a .yaml file
    • code block in YAML format

2021-06-15_10-37-58

Non-breaking API changes have been made to Addon, AddonService and AddonResource to add more info and allow to select the add-on service when managing add-ons. Among other things:

  • Some service-specific properties about the add-on are put in a generic Map<String, Object> (for instance, the number of views and likes as reported by Discourse).
  • The verifiedAuthor field allows a service to mark the author as "trustworthy" - hardcoded to true in the KarafAddonService (the author name being "openHAB") while the marketplace service currently marks every author as "untrustworthy".
    A follow-up overhaul of the add-on management UI would for instance put a "verified" check icon (similarly to social network platforms) next to the author's name and will display trustworthy add-ons prominently.
    Untrusted .jar add-ons might also get a warning that they might not have been properly reviewed, so the user knows installing them is at their own risk.
    The "verified" icon and lack of warning would hopefully incentivize developers to still prefer contributing Java-based add-ons like bindings to the openHAB distribution rather than publish them in the marketplace (or have a dual stable version in the distribution + development versions in the marketplace).
    NOTE: while these UI changes are needed, the marketplace service is currently functional without them.
  • The detailed add-on when retrieved with /rest/addons/{addonId} will also include the entire original post in detailedDescription, which will allow displaying the entire description in the UI, as well as pre-extracted links or code blocks, for instance:

localhost:8080/rest/addons/marketplace:123574?serviceId=marketplace

{
    "author": "Yannick Schaus",
    "backgroundColor": "",
    "configDescriptionURI": "",
    "connection": "",
    "contentType": "application/vnd.openhab.ruletemplate",
    "countries": "",
    "description": "",
    "detailedDescription": "<blockquote>\n<p><strong>Please remove this block after reading these instructions</strong><br>\nYour submission will be reviewed as soon as possible by the staff (moderators and/or admins). They will review the format of the post, but not necessarily the code of the submission itself. If it looks good, the <em>published</em> tag will be added and it will be available in the openHAB UI though the community marketplace add-on service. Only staff can add the <em>published</em> tag but you can self-assess the maturity level tags yourself and add appropriate tags from <em>mature</em>, <em>stable</em>, <em>beta</em> or <em>alpha</em>.</p>\n</blockquote>\n<p><em>[<img src=\"https://community.openhab.org/images/emoji/twitter/crayon.png?v=9\" title=\":crayon:\" class=\"emoji\" alt=\":crayon:\"> Add a screenshot or a logo here. The first image of the post will be promoted seen in the add-on list in the UI.]</em></p>\n<p><em>[<img src=\"https://community.openhab.org/images/emoji/twitter/crayon.png?v=9\" title=\":crayon:\" class=\"emoji\" alt=\":crayon:\"> Replace with your description.]</em></p>\n<h2>\n<a name=\"screenshots-1\" class=\"anchor\" href=\"#screenshots-1\"></a>Screenshots</h2>\n<p><em>[<img src=\"https://community.openhab.org/images/emoji/twitter/crayon.png?v=9\" title=\":crayon:\" class=\"emoji\" alt=\":crayon:\"> Upload screenshots if necessary.]</em></p>\n<h2>\n<a name=\"changelog-2\" class=\"anchor\" href=\"#changelog-2\"></a>Changelog</h2>\n<p><em>[<img src=\"https://community.openhab.org/images/emoji/twitter/crayon.png?v=9\" title=\":crayon:\" class=\"emoji\" alt=\":crayon:\"> Add a list of the changes you made for each version, as suggested below (remove example sections)]</em></p>\n<h3>\n<a name=\"version-02-3\" class=\"anchor\" href=\"#version-02-3\"></a>Version 0.2</h3>\n<ul>\n<li>fixed: …</li>\n<li>added: …</li>\n</ul>\n<h3>\n<a name=\"version-01-4\" class=\"anchor\" href=\"#version-01-4\"></a>Version 0.1</h3>\n<ul>\n<li>initial release</li>\n</ul>\n<h2>\n<a name=\"resources-5\" class=\"anchor\" href=\"#resources-5\"></a>Resources</h2>\n<p><em>[Depending on the content type, you can either add code fences with <code>json</code> or <code>yaml</code> languages, a Gist (<a href=\"http://gist.github.com\">gist.github.com</a>), a GitHub repository, or a link to a .jar file.]</em></p>\n<pre><code class=\"lang-yaml\">uid: 'demo:energymeter'\nlabel: Energy Meter\ndescription: Visualizes the current energy consumption.\nconfigDescriptions:\n  - name: consumption\n    type: TEXT\n    context: item\n    filterCriteria:\n      - name: type\n        value: Number\n    label: Consumption Item\n    description: Data source for current consumption\n    required: true\n  - name: light\n    type: TEXT\n    context: item\n    filterCriteria:\n      - name: type\n        value: Color\n    label: Color Item\n    description: Color light to use for visualisation\n    required: true\n  - name: max\n    type: INTEGER\n    label: Max. consumption\n    description: Maximum value for red light\n    required: true\n    defaultValue: 1500\ntriggers:\n  - id: trigger\n    label: Current consumption changes\n    description: Triggers whenever the current consumption changes its value\n    configuration:\n      itemName: '${consumption}'\n    type: core.ItemStateChangeTrigger\nconditions: []\nactions:\n  - inputs: {}\n    id: setcolor\n    label: Change the light color\n    description: &gt;-\n      Sets the color to a value in the range from green (low consumption) to\n      red (high consumption)\n    configuration:\n      type: application/javascript\n      script: |-\n        var power = newState.toString()\n        var percent = power / (${max} / 100) \n        if(percent &lt; 0) percent = 0\n        var hue = 120 - percent * 1.2\n        events.sendCommand('${light}', hue +',100,100')\n    type: script.ScriptAction\n</code></pre>",
    "id": "marketplace:123575",
    "installed": false,
    "keywords": "",
    "label": "Energy Meter (inline YAML test)",
    "link": "https://community.openhab.org/t/123575",
    "properties": {
        "created_at": "Jun 14, 2021, 11:42:42 AM",
        "like_count": 0,
        "posts_count": 1,
        "tags": [
            "stable"
        ],
        "updated_at": "Jun 14, 2021, 4:43:03 PM",
        "views": 5,
        "yaml_content": "uid: 'demo:energymeter'\nlabel: Energy Meter\ndescription: Visualizes the current energy consumption.\nconfigDescriptions:\n  - name: consumption\n    type: TEXT\n    context: item\n    filterCriteria:\n      - name: type\n        value: Number\n    label: Consumption Item\n    description: Data source for current consumption\n    required: true\n  - name: light\n    type: TEXT\n    context: item\n    filterCriteria:\n      - name: type\n        value: Color\n    label: Color Item\n    description: Color light to use for visualisation\n    required: true\n  - name: max\n    type: INTEGER\n    label: Max. consumption\n    description: Maximum value for red light\n    required: true\n    defaultValue: 1500\ntriggers:\n  - id: trigger\n    label: Current consumption changes\n    description: Triggers whenever the current consumption changes its value\n    configuration:\n      itemName: '${consumption}'\n    type: core.ItemStateChangeTrigger\nconditions: []\nactions:\n  - inputs: {}\n    id: setcolor\n    label: Change the light color\n    description: >-\n      Sets the color to a value in the range from green (low consumption) to\n      red (high consumption)\n    configuration:\n      type: application/javascript\n      script: |-\n        var power = newState.toString()\n        var percent = power / (${max} / 100) \n        if(percent < 0) percent = 0\n        var hue = 120 - percent * 1.2\n        events.sendCommand('${light}', hue +',100,100')\n    type: script.ScriptAction\n"
    },
    "type": "automation",
    "verifiedAuthor": false,
    "version": ""
}

TODO:

  • caching to avoid requesting the Discourse API at every request to /rest/addons?serviceId=marketplace (and allow to force a refresh by using AddonService::refreshSource())

  • plan for when there will be a lot of entries

  • filtering based on maturity level (managed with tags)

  • moderation, there are several options:

    • moderation "a posteriori" i.e. every new topic is published by default, if it's not right a curator can take action like editing or removing the topic
    • moderation via a special published tag (there's a "tag group" feature in Discourse that allows to restrict certain tags on topics to people in a certain group); the topic will be visible on the forum but will not appear in the UI before a curator adds the tags:

    image
    image

    • moderation "a priori" with this option on the categories:

    image
    I'm not sure how it works since my test posts skipped that review phase but I suppose it would appear in https://community.openhab.org/review for reviewers like reports

"Curators" are currently restricted to "staff" meaning admins and moderators as appearing in https://community.openhab.org/about but it could be expanded to:

  • a dedicated user group
  • perhaps all "trust level 3" users (a.k.a. "regulars": https://community.openhab.org/badges/3/regular) - this would reduce administration overhead, since this group is automatically managed and users with this badge can probably reasonably be trusted?
  • This is however unrelated to this PR since it can be decided and configured on Discourse directly

@J-N-K
Copy link
Member

J-N-K commented Jun 15, 2021

A jar might not be sufficient since a lot of bindings have dependencies (on core-features or other bundles). Karaf features should be available as well.

@hmerk
Copy link

hmerk commented Jun 15, 2021

@ghys Great Job Yannick, already saw the new marketplece in the community.

@J-N-K
Copy link
Member

J-N-K commented Jun 15, 2021

kar-files would be another option for more complex addons.

@rkoshak
Copy link

rkoshak commented Jun 15, 2021

I too saw the threads created in the forum and got excited.

A leading question similar to J-N-K's but for rules. Based on the above it appears that only UI created rules will be supported (at least to start). That doesn't seem wholly unreasonable. However, is there any thought about rules/rule templates that have external dependencies too?

For example, I've an implementation of the Time of Day all but ready to submit. In fact I've several implementations of Design Patterns. However, there is a lot of common code between them all and that common code is put into library files in $OH_CONF/automation/lib/javascript/community. Is there any thought made about whether/how to handle that?

I can totally see that being a version two goal but without the ability to include something like that, building and maintaining rule templates like this will become quite a challenge.

@ghys
Copy link
Member Author

ghys commented Jun 16, 2021

Yes and the UI widgets could have dependencies too, static assets like images or stylesheets.
But need to figure out the specs for the packaging and description of all these new use cases: for example a the JSON manifest for an add-on like described in openhab/openhab-addons#8448 could work and describe multiple versions, dependencies to install, etc.

There's a system of "handlers" in this PR which was ported over the old Eclipse Marketplace service (https://github.com/openhab/openhab2-addons/blob/2.5.x/bundles/org.openhab.extensionservice.marketplace/src/main/java/org/openhab/extensionservice/marketplace/MarketplaceExtensionHandler.java); it works similarly: the service submits the Discourse post transformed into an Addon instance to all of them asking whether they can handle it, and one (and one only) should respond "yes it looks that's for me". Then all operations like "is it installed", "install" or "uninstall" are delegated to that handler.

This allowed to port over the existing handlers with little change - the Eclipse Marketplace only supported .jar bundles and rule templates in .json format, and I threw in the UI widget handler + YAML support instead of JSON because it was relatively straightforward. This already handles simple situations when someone just wants to share something easily.

For more complex situations, there could be a handler detecting manifests in the post as mentioned above, that could deal with these complex use cases, but the spec & implementation of this could take time and as you've said it can wait. For the record, don't expect even any of this to land in 3.1.

@kaikreuzer
Copy link
Member

Hey @ghys, really awesome work, thank you so much!
I've tested it (without UI additions) and it works already quite smoothly. Already a decent replacement for the Eclipse marketplace and with even more features already.

For the moderation, I think we could try it with letting people publish directly i.e. moderation "a posteriori". If this leads to much crap and problems, we can still change our approach on it. Nonetheless, using tags to identify published once can be helpful, so people could start drafting and only deciding to publish it once they are satisfied with it.

For @rkoshak's question, I guess that KARs could be solution for it (that could be implemented in a later step). In KARs, we can package feature files, which in turn can package files for the file system or define dependencies to other artefacts that need to be installed with it. Two problems I see with KARs is that dependencies would need to be available in a Maven repo that we reference in the distro and that their creation is not as straight forward as zipping an archive with a few files only. But I guess whenever one wants to introduce dependency management, complexity naturally comes with it... That's a good argument to leave it out in step 1.

What do you still see as a todo on this PR? The todos you list above seem to be "nice to haves" for a follow-up, but not necessarily mandatory for the first implementation, right?

@rkoshak
Copy link

rkoshak commented Jun 16, 2021

As long as the process is documented well enough for those who don't know maven from makefiles and/or provide tooling and scripts, I don't think putting them into kar files is too much to ask.

Though in this scenario each kar file is self contained, or can they define external stuff as dependencies. For example, let's say I write a rule template in Python. Obviously that means the rule will require the Jython add-on as a requirement. Can I specify that requirement in the kar file somehow? If not it can be handled in the docs for the rule template I'm sure.

Another example. Let's say I've written a utility library called timerMgr which makes it easier to create and interact with a bunch of timers in a rule (e.g. a one timer per Item situation). This isn't a rule unto itself but used by rules. So would I deploy this as it's own kar file and then in my Time of Day Rule just specify "this rule depends on timerMgr" or would I embed timerMgr into the Time of Day kar file and make Time of Day available as a self contained unit?

I don't see anything wrong with either approach. The rules developer doesn't necessarily need to keep multiple copies of timerMgr in their source control system. The kar build process would just have to grab a copy of it when building the kar file for Time Of Day and any other rule that uses it.

I'm just thinking about the thought process so I can start preparing my existing code and structuring new stuff I write. I don't mind being an alfa and beta tester for this even if it means making inline copies of my libraries into each rule for now. I'm sure I can come up with a script to make that easier to manage until version 2 and 3 when kar files (or whatever) becomes supported. :-D But it would be nice to plan now for what's coming down the line.

@ghys
Copy link
Member Author

ghys commented Jun 17, 2021

For the moderation, I think we could try it with letting people publish directly i.e. moderation "a posteriori". If this leads to much crap and problems, we can still change our approach on it. Nonetheless, using tags to identify published once can be helpful, so people could start drafting and only deciding to publish it once they are satisfied with it.

Agreed, let's go with the tag then; we can either educate authors so they apply it themselves, this allows an eventual drafting phase before self-publishing (and even unpublishing if necessary); and if no pre-moderation leads to vandalism, abuse or poor quality entries we have the ability to change policy instantly and restrict self-publishing using the "tags are visible to anyone but only the following groups can use them" option shown above.
There's no HTML parsing happening when getting the list of add-ons so this would be pretty safe, even if a post is completely wrong, the "list" API would still work and choke only when clicking on that particular add-on to see the details.

What do you still see as a todo on this PR? The todos you list above seem to be "nice to haves" for a follow-up, but not necessarily mandatory for the first implementation, right?

  • I was on the fence about caching of results locally with a refresh job and/or force refresh option similarly to your MarketplaceProxy but then I ended up not doing it as I thought it's fast enough and doesn't cause more overhead on the Discourse server than simply browsing the category (it "scrolls" automatically and fetches the next page automatically until the end though but it would only become a problem when we get more entries). Wdyt?

  • Unit tests would be nice especially wrt. post parsing, I ran into gotchas by accident like & being escaped to &amp;...

  • Support for maturity levels with tags would be a low-hanging fruit. Maybe licenses too, although authors can simply specify the license & eventual links to source code etc. directly in their post.

Mostly I submitted this draft PR as soon as I had something working to validate some of the design choices and possibly adjust them early, like:

  • part of the core instead of an add-on?
  • part of the openhab-runtime-base feature? (third-party distributions can simply leave it out)
  • perhaps add a "Enable Community Marketplace" master switch in Settings > Add-on Management like the existing "Access Remote Repository", in which case would it be opt-in or opt-out?

@kaikreuzer
Copy link
Member

I ended up not doing it as I thought it's fast enough and doesn't cause more overhead on the Discourse server than simply browsing the category

I think the same, that's why I meant caching could be done at a later stage in case we see that we need it. As we roll it out slowly (first snapshots, then milestones and only at the end of 2021 as a release), I think we have enough time to check what load it causes in reality.

Licenses are a good point, I think that is a must. We might even have closed source stuff, where people only provide a jar. Would hence be good to possibly have both, a license as a SPDX and a link to the license text.

part of the core instead of an add-on?

Yes, I am for it. It does not add much to the distro and improves the UX and your possibilities in the UI.

part of the openhab-runtime-base feature?

Yes, see above.

perhaps add a "Enable Community Marketplace" master switch in Settings > Add-on Management

Yes, an opt-out makes sense if people want to really have an offline system.

@cweitkamp cweitkamp added the enhancement An enhancement or new feature of the Core label Jul 21, 2021
@kaikreuzer
Copy link
Member

perhaps add a "Enable Community Marketplace" master switch in Settings > Add-on Management

Yes, an opt-out makes sense if people want to really have an offline system.

I just had a look at how to implement this. Unfortunately, the "Add-on Management" service (and with it the addons.cfg file) is highly coupled to the Karaf feature installer and it won't be easy to refactor this.
It might be easier to introduce a dedicated system service id org.openhab.marketplace with dedicated settings for whatever we want to introduce as configuration for the marketplace. Would that make sense to you?

@J-N-K
Copy link
Member

J-N-K commented Aug 8, 2021

Some bindings (e.g. SNMP, Network, ...) need global configuration. Dropping the JAR of such a binding in the addons folder starts the binding, but the configuration can only be adjusted manually on the karaf console via the configuration admin (which is nearly impossible to explain to the "normal" user) because the UI only shows addons that are installed as a feature. This should be avoided for marketplace addons.

@ghys
Copy link
Member Author

ghys commented Aug 8, 2021

I just had a look at how to implement this. Unfortunately, the "Add-on Management" service (and with it the addons.cfg file) is highly coupled to the Karaf feature installer and it won't be easy to refactor this.
It might be easier to introduce a dedicated system service id org.openhab.marketplace with dedicated settings for whatever we want to introduce as configuration for the marketplace. Would that make sense to you?

Too bad, but having a separate service configuration works. I was merely advocating for a consolidation of the configuration of similar services to avoid a proliferation of top-level menu entries in the UI with only a couple of options. Regional Settings and Ephemeris also come to mind - I remember suggesting this a while ago. But if it's not trivial for a service to source its configuration from another service, then so be it. This could be solved at the UI level too, I guess.

Dropping the JAR of such a binding in the addons folder starts the binding, but the configuration can only be adjusted manually on the karaf console via the configuration admin (which is nearly impossible to explain to the "normal" user) because the UI only shows addons that are installed as a feature. This should be avoided for marketplace addons.

This has been reported before and thus was added to the page where you can choose the binding when adding a new thing (Settings > Things > + button). Configurable bindings will have a gears icon you can click to access the configuration. It's the only place where all bindings whatever their source are listed. To make a comparison with e.g. Android, the Settings > Add-ons part of the UI is the Play Store (so mostly used to browse and install/uninstall) while dropping a .jar in the addons folder is like sideloading an app from its .apk file.
Maybe there could be a "local" add-on source to determine which add-ons are installed in this way (there has been some discussion about extending the "configurability" to all types of add-ons anyway, not only bindings). The Marketplace add-on source supports determining which add-ons are installed so they will appear in Settings > Add-ons, and there's hope people who distributed their .jars on the forum to drop in the addons folder will use the Marketplace instead, so the experience would be more streamlined for users.

@J-N-K
Copy link
Member

J-N-K commented Aug 8, 2021

@ghys Interesting. I never noticed that. Thanks.

ghys added a commit to ghys/openhab-webui that referenced this pull request Sep 7, 2021
Depends on openhab/openhab-core#2405.

This replaces the current section of the UI for managing
add-ons with a completely new "store-like" interface.
External data, like logos and detailed descriptions for
distribution add-ons are fetched from well-known locations
on the website or GitHub to provide a "store" look & feel.

The add-on store is orgnanized around 4 categories: bindings,
automation, UI, and others, with a separate search tab.
Each category is splitted into subsections. The availability
of these sections depend on the add-on services that are
registered. The sections currently supports add-ons provided
by the "karaf" & "marketplace" services, as provided by
openhab/openhab-core#2405.

Signed-off-by: Yannick Schaus <[email protected]>
@kaikreuzer
Copy link
Member

@ghys As you have now created openhab/openhab-webui#1158 that is waiting on this PR - will you soon remove its "draft" status, so that we can do a final review? As discussed above, I don't really see any todo that still needs to be added to this initial PR.

@ghys
Copy link
Member Author

ghys commented Sep 8, 2021

Yes I still have minor quirks with it but it will be ready for prime time soon.

ghys added 6 commits September 9, 2021 20:47
Signed-off-by: Yannick Schaus <[email protected]>
Signed-off-by: Yannick Schaus <[email protected]>
Change content types

Javadoc fixes

Implement getAddId(URL) for marketplace (unused)

Bump Maven version

Signed-off-by: Yannick Schaus <[email protected]>
Change content type for bundles

Signed-off-by: Yannick Schaus <[email protected]>
@ghys ghys marked this pull request as ready for review September 9, 2021 19:48
@ghys ghys requested a review from a team as a code owner September 9, 2021 19:48
@kaikreuzer
Copy link
Member

SAT failed with

[ERROR] org.openhab.core.addon.marketplace.MarketplaceBundleInstaller.java:[92]
Count of placeholder(0) is not equal to count of parameter(1)

Copy link
Member

@kaikreuzer kaikreuzer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ghys, your updates look awesome! I think this is now a pretty good solution for the add-on bundles as we also store the content of UI widgets and rule templates locally in USER_DATA and don't fetch newer versions of them after an openHAB update - so all types are now nicely aligned in the way they are handled.

Once the SAT issue is fixed, we should be good to merge!

Signed-off-by: Yannick Schaus <[email protected]>
@ghys
Copy link
Member Author

ghys commented Sep 13, 2021

Btw, I just noticed that with the setting of the "location id", you can use the marketplace add-on ID to address the related bundle in the Karaf console, which is kind of cool I guess:

openhab> bundle:list -n -u
215 │ Active │  75 │ 3.2.0.202108282150    │ mvn:org.openhab.core.bundles/org.openhab.core.thing.xml/3.2.0-SNAPSHOT
216 │ Active │  80 │ 3.2.0.202108282150    │ mvn:org.openhab.core.bundles/org.openhab.core.transform/3.2.0-SNAPSHOT
217 │ Active │  80 │ 3.2.0.202108282152    │ mvn:org.openhab.core.bundles/org.openhab.core.ui/3.2.0-SNAPSHOT
218 │ Active │  80 │ 3.2.0.202108282156    │ mvn:org.openhab.core.bundles/org.openhab.core.ui.icon/3.2.0-SNAPSHOT
219 │ Active │  80 │ 3.2.0.202108282150    │ mvn:org.openhab.core.bundles/org.openhab.core.voice/3.2.0-SNAPSHOT
...
234 │ Active │  80 │ 2.1.0                 │ mvn:tech.uom.lib/uom-lib-common/2.1
235 │ Active │  80 │ 1.4.17                │ mvn:com.thoughtworks.xstream/xstream/1.4.17
236 │ Active │  80 │ 3.1.0.202103111938    │ marketplace:123568
openhab> bundle:info marketplace:123568

openHAB Add-ons :: Bundles :: yamahamusiccast Binding (236)
-----------------------------------------------------------
openhab> bundle:stop marketplace:123568
openhab> bundle:status marketplace:123568
Resolved
openhab> bundle:start marketplace:123568
openhab> bundle:status marketplace:123568
Active

@kaikreuzer kaikreuzer added rebuild Triggers the Jenkins PR build and removed rebuild Triggers the Jenkins PR build labels Sep 14, 2021
@kaikreuzer kaikreuzer merged commit 2663a3f into openhab:main Sep 14, 2021
@kaikreuzer kaikreuzer added this to the 3.2 milestone Sep 14, 2021
@J-N-K
Copy link
Member

J-N-K commented Sep 16, 2021

Unfortunately - contrary to the statement in the first post - the changes to org.openhab.core.addon.Addon are breaking, since existing constructors where removed.

@ghys
Copy link
Member Author

ghys commented Sep 16, 2021

You're right, shouldn't be a problem to add overloads with the old signatures. Mistakes happen.

@J-N-K
Copy link
Member

J-N-K commented Sep 16, 2021

I'm already preparing a PR for that.

  • author -> unknown
  • authorVerified -> false
  • contentType -> ?
  • all other fields -> null

is "unknown" ok for the contentType?

@ghys
Copy link
Member Author

ghys commented Sep 16, 2021

is "unknown" ok for the contentType?

I guess, or even the empty string or null.
It's supposed to be a media type like application/java-archive for generic .jar archives.
(vendor-specific media types are defined in the PR, for example application/vnd.openhab.bundle for individual bundles or application/vnd.openhab.feature;type=karaf for Karaf features - there aren't IANA-approved media types for these).

May I ask where is the old API used outside OHC? Because if it's third-party and you're relying on the current generic add-on management UI, it will definitely break with openhab/openhab-webui#1158 (but I will keep the current codebase in the repo and if all goes well it would be relatively easy for forks to revert to it, with a compilation flag or similar).

@J-N-K
Copy link
Member

J-N-K commented Sep 16, 2021

Yes, it's 3rd party. It's something similar to the marketplace (manager for installing 3rd party addons).

Unfortunately it can't be made obsolete (which I would prefer) by the marketplace because the marketplace only supports .jar, neither complete features (which I would prefer) nor .kar.

If it'll break anyway, there is no need to re-add it. I'll just provide another version for OH 3.2.

@ghys
Copy link
Member Author

ghys commented Sep 16, 2021

because the marketplace only supports .jar, neither complete features (which I would prefer) nor .kar.

This could be solved with another MarketplaceAddonHandler implementation, and I figure it could benefit the openHAB project as well. It's just not clear enough currently how to contribute full-fledged features unless via PR to openhab-addons (it might be a technical matter as well as as an "developer education" matter).

@wborn
Copy link
Member

wborn commented Sep 18, 2021

Hello @ghys I have a small question. 🙂

Was there a reason for not already adding the other add-on types (io/misc, persistence, transform, voice)?:

types.put(BINDINGS_CATEGORY, new AddonType("binding", "Bindings"));
types.put(RULETEMPLATES_CATEGORY, new AddonType("automation", "Automation"));
types.put(UIWIDGETS_CATEGORY, new AddonType("ui", "User Interfaces"));
contentTypes.put("binding", BINDINGS_CONTENT_TYPE);
contentTypes.put("automation", RULETEMPLATES_CONTENT_TYPE);
contentTypes.put("ui", UIWIDGETS_CONTENT_TYPE);

@ghys
Copy link
Member Author

ghys commented Sep 18, 2021

no real reason, the goal was to restore the functionality of the Eclipse Marketplace. Also to avoid having to create empty Discourse categories.

currently you can infer the type and content type from the Discourse category alone:

  • Bindings category => binding add-on type => application/vnd.openhab. bundle
  • Rule Templates category => automation add-on => application/vnd.openhab.ruletemplate
  • UI Widgets category => ui add-on => application/vnd.openhab.uicomponent;type=widget

But this logic obviously has to change now. Maybe put all Java based add-ons in one category (rename Bindings to Bundles & Features or similar?) and use mandatory tags ("transformation" , "kar") to derive the type and content type.

@J-N-K
Copy link
Member

J-N-K commented Sep 18, 2021

Is it possible to have mandatory tags with "one-of-list"? So we can force to set one of kar, jar, json, yaml and also one of a list of categories? That would also solve the issue discussed in my PR with finding the correct content-type, which would then be set according to the tag.

@wborn
Copy link
Member

wborn commented Sep 18, 2021

Yes the content type is a bit difficult to handle with the categories. It is also likely that there will be Automation (script engines) and UI add-on JARs or KARs. Those don't fit the Bindings category.

@ghys
Copy link
Member Author

ghys commented Sep 18, 2021

Is it possible to have mandatory tags with "one-of-list"? So we can force to set one of kar, jar, json, yaml and also one of a list of categories?

Possibly, check out the screenshot in the first post about tag groups, there's an option that does that apparently.
Anyway if there's a problem with tags, there's always the option to ignore the entry.

@J-N-K
Copy link
Member

J-N-K commented Sep 18, 2021

Not sure if you can make tags mandatory, but at least we can prevent invalid combinations (like kar and jar for the same addon) with the tag groups. Nice. Looks like an easy way. If we agree on that, I can make the changes in the kar handler PR.

@ghys
Copy link
Member Author

ghys commented Sep 18, 2021

You can, but unfortunately only from one tag group.

image

Maybe we can consider .jar to be the default (as it's the easiest to make) and .kar would need a "kar" tag. In other terms no "kar" tag means .jar.

ghys added a commit to openhab/openhab-webui that referenced this pull request Sep 24, 2021
Depends on openhab/openhab-core#2405.

This replaces the current section of the UI for managing
add-ons with a completely new "store-like" interface.
External data, like logos and detailed descriptions for
distribution add-ons are fetched from well-known locations
on the website or GitHub to provide a "store" look & feel.

The add-on store is orgnanized around 4 categories: bindings,
automation, UI, and others, with a separate search tab.
Each category is splitted into subsections. The availability
of these sections depend on the add-on services that are
registered. The sections currently supports add-ons provided
by the "karaf" & "marketplace" services, as provided by
openhab/openhab-core#2405.
Other add-on services should be supported as well.

Signed-off-by: Yannick Schaus <[email protected]>
wborn added a commit to wborn/openhab-distro that referenced this pull request Sep 25, 2021
kaikreuzer pushed a commit to openhab/openhab-distro that referenced this pull request Sep 25, 2021
splatch pushed a commit to ConnectorIO/copybara-hab-core that referenced this pull request Jul 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement An enhancement or new feature of the Core
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants