gitea_admin/sientia-dataops-laborious_temporal
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

SIENTIAPDE-1273

Browse Source 
Refactor data handling in various modules to ensure DataFrame consistency

- Replaced direct DataFrame instantiation with `ensure_dataframe` utility in Gates, MLFlow, OPC, and ModelMetrics classes to standardize data handling.
- Updated return types in several asynchronous methods to return DataFrames instead of dictionaries for improved usability.
- Adjusted data export processes in workflows to convert DataFrames to dictionaries with `to_dict(orient='records')` for compatibility with downstream systems.
This commit is contained in:
vitor-aignosi
2025-11-14 16:55:00 -03:00
parent 4191f68d3b
commit d2b365a34d
12 changed files with 368 additions and 28 deletions
Download Patch File Download Diff File Expand all files Collapse all files

151
tests.ipynb
View File

@@ -588,6 +588,157 @@
"\n",
"display(prediction_data)"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "486b95b3",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>a</th>\n",
" <th>b</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>2025-01-01</th>\n",
" <td>1</td>\n",
" <td>4</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2025-01-02</th>\n",
" <td>2</td>\n",
" <td>5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2025-01-03</th>\n",
" <td>3</td>\n",
" <td>6</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" a b\n",
"2025-01-01 1 4\n",
"2025-01-02 2 5\n",
"2025-01-03 3 6"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/plain": [
"{'index': ['2025-01-01', '2025-01-02', '2025-01-03'],\n",
" 'columns': ['a', 'b'],\n",
" 'data': [[1, 4], [2, 5], [3, 6]]}"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>0</th>\n",
" <th>1</th>\n",
" <th>2</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>index</th>\n",
" <td>2025-01-01</td>\n",
" <td>2025-01-02</td>\n",
" <td>2025-01-03</td>\n",
" </tr>\n",
" <tr>\n",
" <th>columns</th>\n",
" <td>a</td>\n",
" <td>b</td>\n",
" <td>None</td>\n",
" </tr>\n",
" <tr>\n",
" <th>data</th>\n",
" <td>[1, 4]</td>\n",
" <td>[2, 5]</td>\n",
" <td>[3, 6]</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" 0 1 2\n",
"index 2025-01-01 2025-01-02 2025-01-03\n",
"columns a b None\n",
"data [1, 4] [2, 5] [3, 6]"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from pandas import DataFrame\n",
"\n",
"data = DataFrame({\n",
" \"a\": {\"2025-01-01\": 1, \"2025-01-02\": 2, \"2025-01-03\": 3},\n",
" \"b\": {\"2025-01-01\": 4, \"2025-01-02\": 5, \"2025-01-03\": 6},\n",
"})\n",
"\n",
"display(data)\n",
"\n",
"data_list = data.to_dict('split')\n",
"\n",
"display(data_list)\n",
"\n",
"data_rec = DataFrame.from_dict(data_list, orient='index')\n",
"\n",
"display(data_rec)"
]
}
],
"metadata": {