Skip to content

Commit dab71ce

Browse files
dfvnetodfvneto
authored
Merge pull request #404 from platiagro/develop
Bump to Version 0.3.1
2 parents cef2c44 + 104209d commit dab71ce

22 files changed

+643
-182
lines changed

‎README.md

+4-5
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,6 @@ environment variables:
5555
MAIL_SERVER hostname of a smtp service (default: ).
5656
MAIL_USERNAME username of a smtp service (default: ).
5757
MAIL_PASSWORD password of a smtp service (default: ).
58-
SHARE_TASK_CONTAINER_IMAGE docker image used in task sharing jobs (default: platiagro/share-task:0.3.0-SNAPSHOT).
5958
MAIL_SENDER_ADDRESS sender address for emails sent by the smtp service (default: ).
6059
SELDON_REST_TIMEOUT response timeout in milliseconds for seldondeployments (default: 60000)
6160
SELDON_LOGGER_ENDPOINT logger service URL that receives seldondeployment responses (default: http://projects.platiagro:8080)
@@ -71,7 +70,7 @@ environment variables:
7170
**Using Docker**
7271

7372
```bash
74-
docker build -t platiagro/projects:0.3.0-SNAPSHOT -f Dockerfile .
73+
docker build -t platiagro/projects:0.3.1-SNAPSHOT -f Dockerfile .
7574
```
7675

7776
Example:
@@ -87,7 +86,7 @@ docker run -it -p 8080:8080 \
8786
--env "MYSQL_DB_USER=$MYSQL_DB_USER" \
8887
--env "MYSQL_DB_PASSWORD=$MYSQL_DB_PASSWORD" \
8988
--env "KF_PIPELINES_ENDPOINT=$KF_PIPELINES_ENDPOINT" \
90-
platiagro/projects:0.3.0-SNAPSHOT
89+
platiagro/projects:0.3.1-SNAPSHOT
9190
```
9291

9392
## Persistence Agent
@@ -115,7 +114,7 @@ optional arguments:
115114
**Using Docker**
116115
117116
```bash
118-
docker build -t platiagro/persistence-agent:0.3.0-SNAPSHOT -f Dockerfile.persistenceagent .
117+
docker build -t platiagro/persistence-agent:0.3.1-SNAPSHOT -f Dockerfile.persistenceagent .
119118
```
120119
121120
```bash
@@ -126,7 +125,7 @@ docker run -it \
126125
--env "MYSQL_DB_NAME=$MYSQL_DB_NAME" \
127126
--env "MYSQL_DB_USER=$MYSQL_DB_USER" \
128127
--env "MYSQL_DB_PASSWORD=$MYSQL_DB_PASSWORD" \
129-
platiagro/persistence-agent:0.3.0-SNAPSHOT
128+
platiagro/persistence-agent:0.3.1-SNAPSHOT
130129
```
131130
132131
## Testing

‎docs/swagger.yaml

+71-39
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
openapi: 3.0.0
22
info:
33
title: PlatIAgro Projects API
4-
version: 0.3.0
4+
version: 0.3.1
55
description: >
66
These are the docs for PlatIAgro Projects API. The endpoints below are
77
usually accessed by the PlatIAgro Web-UI.
@@ -96,34 +96,15 @@ paths:
9696
CannotConnectToDatabase:
9797
$ref: '#/components/examples/CannotConnectToDatabase'
9898
/tasks:
99-
get:
100-
summary: 'List tasks. Supports pagination, and sorting.'
99+
post:
100+
summary: Create a new task.
101101
tags:
102102
- Tasks
103-
parameters:
104-
- in: query
105-
name: name
106-
schema:
107-
type: string
108-
description: Name filter
109-
- in: query
110-
name: page
111-
schema:
112-
type: integer
113-
description: Page number
114-
- in: query
115-
name: page_size
116-
schema:
117-
type: integer
118-
description: Page size
119-
- in: query
120-
name: order
121-
schema:
122-
type: string
123-
example: uuid asc
103+
requestBody:
104+
$ref: '#/components/requestBodies/TaskPost'
124105
responses:
125106
'200':
126-
$ref: '#/components/responses/Tasks'
107+
$ref: '#/components/responses/Task'
127108
'400':
128109
description: >
129110
Bad Request response status code indicates that the server cannot or
@@ -135,21 +116,28 @@ paths:
135116
schema:
136117
$ref: '#/components/schemas/BadRequest'
137118
examples:
138-
InvalidOrderBy:
139-
$ref: '#/components/examples/InvalidOrderBy'
119+
MissingRequiredNotebookOrTaskId:
120+
$ref: '#/components/examples/MissingRequiredNotebookOrTaskId'
121+
TaskNameExists:
122+
$ref: '#/components/examples/TaskNameExists'
123+
InvalidTaskId:
124+
$ref: '#/components/examples/InvalidTaskId'
125+
InvalidCategory:
126+
$ref: '#/components/examples/InvalidCategory'
140127
'500':
141128
$ref: '#/components/responses/InternalServerError'
142129
'503':
143130
$ref: '#/components/responses/ServiceUnavailable'
131+
/tasks/list-tasks:
144132
post:
145-
summary: Create a new task.
133+
summary: 'List tasks. Supports pagination, and sorting.'
146134
tags:
147135
- Tasks
148136
requestBody:
149-
$ref: '#/components/requestBodies/TaskPost'
137+
$ref: '#/components/requestBodies/ListTasks'
150138
responses:
151139
'200':
152-
$ref: '#/components/responses/Task'
140+
$ref: '#/components/responses/Tasks'
153141
'400':
154142
description: >
155143
Bad Request response status code indicates that the server cannot or
@@ -161,14 +149,12 @@ paths:
161149
schema:
162150
$ref: '#/components/schemas/BadRequest'
163151
examples:
164-
MissingRequiredNotebookOrTaskId:
165-
$ref: '#/components/examples/MissingRequiredNotebookOrTaskId'
166-
TaskNameExists:
167-
$ref: '#/components/examples/TaskNameExists'
168-
InvalidTaskId:
169-
$ref: '#/components/examples/InvalidTaskId'
170-
InvalidCategory:
171-
$ref: '#/components/examples/InvalidCategory'
152+
InvalidOrderBy:
153+
$ref: '#/components/examples/InvalidOrderBy'
154+
NotAllowedCharacter:
155+
$ref: '#/components/examples/NotAllowedCharacter'
156+
ExceededCharacterAmount:
157+
$ref: '#/components/examples/ExceededCharacterAmount'
172158
'500':
173159
$ref: '#/components/responses/InternalServerError'
174160
'503':
@@ -382,7 +368,7 @@ paths:
382368
$ref: '#/components/responses/InternalServerError'
383369
'503':
384370
$ref: '#/components/responses/ServiceUnavailable'
385-
/projects/listprojects:
371+
/projects/list-projects:
386372
post:
387373
summary: 'List projects. Supports pagination, and sorting.'
388374
tags:
@@ -405,6 +391,10 @@ paths:
405391
examples:
406392
InvalidOrderBy:
407393
$ref: '#/components/examples/InvalidOrderBy'
394+
NotAllowedCharacter:
395+
$ref: '#/components/examples/NotAllowedCharacter'
396+
ExceededCharacterAmount:
397+
$ref: '#/components/examples/ExceededCharacterAmount'
408398
'500':
409399
$ref: '#/components/responses/InternalServerError'
410400
'503':
@@ -710,6 +700,23 @@ paths:
710700
responses:
711701
'200':
712702
$ref: '#/components/responses/Experiments'
703+
'400':
704+
description: >
705+
Bad Request response status code indicates that the server cannot or
706+
will not process the request due to something that is perceived to
707+
be a client error. A common cause is that the client has sent
708+
invalid request values.
709+
content:
710+
application/json:
711+
schema:
712+
$ref: '#/components/schemas/BadRequest'
713+
examples:
714+
InvalidOrderBy:
715+
$ref: '#/components/examples/InvalidOrderBy'
716+
NotAllowedCharacter:
717+
$ref: '#/components/examples/NotAllowedCharacter'
718+
ExceededCharacterAmount:
719+
$ref: '#/components/examples/ExceededCharacterAmount'
713720
'404':
714721
description: >
715722
Not Found client error response code indicates that the server can't
@@ -3975,6 +3982,23 @@ components:
39753982
type: number
39763983
h:
39773984
type: number
3985+
ListTasks:
3986+
content:
3987+
application/json:
3988+
schema:
3989+
type: object
3990+
properties:
3991+
filters:
3992+
type: object
3993+
properties:
3994+
name:
3995+
type: string
3996+
order:
3997+
type: string
3998+
page:
3999+
type: integer
4000+
page_size:
4001+
type: integer
39784002
TaskPost:
39794003
content:
39804004
application/json:
@@ -4677,6 +4701,14 @@ components:
46774701
value:
46784702
code: InvalidOrderBy
46794703
message: Invalid order argument
4704+
NotAllowedCharacter:
4705+
value:
4706+
code: NotAllowedChar
4707+
message: Character not Allowed
4708+
ExceededCharacterAmount:
4709+
value:
4710+
code: ExceededCharQuantity
4711+
message: Exceeded maximum character quantity allowed
46804712
ProjectNameExists:
46814713
value:
46824714
code: ProjectNameExists

‎projects/__init__.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
__version__ = "0.3.0"
1+
__version__ = "0.3.1"

‎projects/api/projects.py

+2-2
Original file line numberDiff line numberDiff line change
@@ -14,14 +14,14 @@
1414
)
1515

1616

17-
@router.post("/listprojects", response_model=projects.schemas.project.ProjectList)
17+
@router.post("/list-projects", response_model=projects.schemas.project.ProjectList)
1818
async def handle_list_projects(
1919
request_schema: projects.schemas.project.ProjectListRequest,
2020
session: Session = Depends(database.session_scope),
2121
kubeflow_userid: Optional[str] = Header(database.DB_TENANT),
2222
):
2323
"""
24-
Handles POST requests to /listprojects.
24+
Handles POST requests to /list-projects.
2525
Parameters
2626
----------
2727
request_schema : projects.schemas.project.ProjectListRequest

‎projects/api/tasks/tasks.py

+2-2
Original file line numberDiff line numberDiff line change
@@ -94,7 +94,7 @@ async def handle_get_task(
9494
@router.patch("/{task_id}", response_model=projects.schemas.task.Task)
9595
async def handle_patch_task(
9696
task_id: str,
97-
task: projects.schemas.task.TaskUpdate,
97+
task: projects.schemas.task.TaskCreate,
9898
background_tasks: BackgroundTasks,
9999
session: Session = Depends(database.session_scope),
100100
):
@@ -104,7 +104,7 @@ async def handle_patch_task(
104104
Parameters
105105
----------
106106
task_id : str
107-
task : projects.schemas.task.TaskUpdate
107+
task : projects.schemas.task.TaskCreate
108108
background_tasks : fastapi.BackgroundTasks
109109
session : sqlalchemy.orm.session.Session
110110

‎projects/config/Deployment.ipynb

+24-8
Original file line numberDiff line numberDiff line change
@@ -27,25 +27,41 @@
2727
"outputs": [],
2828
"source": [
2929
"%%writefile Model.py\n",
30-
"# adicione imports aqui...\n",
30+
"# adicione imports necessários aqui...\n",
31+
"import joblib\n",
32+
"import pandas as pd\n",
3133
"\n",
3234
"class Model:\n",
3335
" def __init__(self):\n",
3436
" self.loaded = False\n",
3537
"\n",
3638
" def load(self):\n",
37-
" # utilize esta função para carregar e inicializar modelos\n",
38-
"\n",
39+
" # utilize esta função para carregar e inicializar modelos salvos anteriormente\n",
40+
" \n",
41+
" artifacts = joblib.load(\"/tmp/data/nova-tarefa.joblib\")\n",
42+
" self.pipeline = artifacts[\"pipeline\"]\n",
43+
" self.columns_to_filter = artifacts[\"columns_to_filter\"]\n",
3944
" self.loaded = True\n",
4045
" print(\"Loaded model\")\n",
4146
"\n",
4247
" def predict(self, X, feature_names, meta=None):\n",
48+
" # a função predict recebe um parâmetro X (numpy array, str ou bytes)\n",
49+
" # e deve retornar o resultado da tarefa (numpy array, list, str ou bytes)\n",
50+
" \n",
4351
" if not self.loaded:\n",
4452
" self.load()\n",
53+
" \n",
54+
" df = pd.DataFrame(X)\n",
55+
" \n",
56+
" if feature_names:\n",
57+
" # Antes de utilizar o conjunto de dados X no modelo, reordena suas features de acordo com a ordem utilizada no treinamento\n",
58+
" df = pd.DataFrame(X, columns=feature_names)\n",
59+
" df = df[self.columns_to_filter]\n",
4560
"\n",
46-
" # a função predict recebe um parâmetro X (numpy array, str ou bytes)\n",
47-
" # e deve retornar o resultado da tarefa (numpy array, list, str ou bytes)\n",
48-
" return X"
61+
" # Realiza inferência\n",
62+
" y_pred = self.pipeline.predict(df)\n",
63+
"\n",
64+
" return y_pred"
4965
]
5066
}
5167
],
@@ -66,9 +82,9 @@
6682
"name": "python",
6783
"nbconvert_exporter": "python",
6884
"pygments_lexer": "ipython3",
69-
"version": "3.7.8"
85+
"version": "3.8.8"
7086
}
7187
},
7288
"nbformat": 4,
7389
"nbformat_minor": 4
74-
}
90+
}

‎projects/config/Experiment.ipynb

+16-5
Original file line numberDiff line numberDiff line change
@@ -93,7 +93,7 @@
9393
"## Salva resultados da tarefa\n",
9494
"\n",
9595
"A plataforma guarda o conteúdo de `/tmp/data/` para as tarefas subsequentes.<br>\n",
96-
"Use essa pasta para salvar modelos, metadados e outros resultados."
96+
"Use essa pasta para salvar modelos, metadados e outros resultados que serão usados no Deployment."
9797
]
9898
},
9999
{
@@ -102,13 +102,24 @@
102102
"metadata": {},
103103
"outputs": [],
104104
"source": [
105-
"with open(\"/tmp/data/outputs.txt\", \"w\") as f:\n",
106-
" f.write(\"...\")"
105+
"from joblib import dump\n",
106+
"\n",
107+
"artifacts = {\n",
108+
" \n",
109+
" # exemplos\n",
110+
" # \"nome_resultado\": variavel_do_resultado\n",
111+
" \n",
112+
" \"pipeline\": pipeline,\n",
113+
" \"columns_to_filter\": columns_to_filter,\n",
114+
"}\n",
115+
"\n",
116+
"dump(artifacts, \"/tmp/data/nova-tarefa.joblib\")"
107117
]
108118
}
109119
],
110120
"metadata": {
111121
"celltoolbar": "Tags",
122+
112123
"kernelspec": {
113124
"display_name": "Python 3",
114125
"language": "python",
@@ -124,9 +135,9 @@
124135
"name": "python",
125136
"nbconvert_exporter": "python",
126137
"pygments_lexer": "ipython3",
127-
"version": "3.7.8"
138+
"version": "3.8.8"
128139
}
129140
},
130141
"nbformat": 4,
131142
"nbformat_minor": 4
132-
}
143+
}

0 commit comments

Comments
 (0)