Release 11 (from main)

index.encoded.txt

@@ -1 +1 @@
-store_name: SUSE Store<lb>version: 10<lb>models:<lb>- name: linear_regression<lb>  version: 1.2.0<lb>- name: xgboost<lb>  version: 1.2.0<lb>runtimes:<lb>  basic:<lb>    path: runtime/basic.yaml<lb>    version: 7<lb>  single:<lb>    path: runtime/single.yaml<lb>    version: 1<lb>  xgboost:<lb>    path: runtime/xgboost.yaml<lb>    version: 7<lb>
+store_name: SUSE Store<lb>version: 11<lb>models:<lb>- name: linear_regression<lb>  version: 1.2.2<lb>- name: xgboost<lb>  version: 1.2.1<lb>runtimes:<lb>  basic:<lb>    path: runtime/basic.yaml<lb>    version: 8<lb>  single:<lb>    path: runtime/single.yaml<lb>    version: 1<lb>  xgboost:<lb>    path: runtime/xgboost.yaml<lb>    version: 8<lb>

index.yaml

@@ -1,17 +1,17 @@
 store_name: SUSE Store
-version: 10
+version: 11
 models:
 - name: linear_regression
-  version: 1.2.0
+  version: 1.2.2
 - name: xgboost
-  version: 1.2.0
+  version: 1.2.1
 runtimes:
   basic:
     path: runtime/basic.yaml
-    version: 7
+    version: 8
   single:
     path: runtime/single.yaml
     version: 1
   xgboost:
     path: runtime/xgboost.yaml
-    version: 7
+    version: 8

models/linear_regression/README.md

@@ -23,7 +23,7 @@ These parameters are defined in the `model.yaml` under `data_model.input_map` an
 | `scaler__name`       | `'Standard Scaler'`                                                                                                      | Name of the scaler to use for scaling the data. Options: 'Standard Scaler' |
 | `verbose`            | `true`                                                                                                                   | Enable verbose output during model execution.                              |
 | `steps_order`        | `['Discontinuity Treatment', 'Lag Selection', 'Range Selection & Data Removal', 'Static Window Removal', 'Define Variables Limits', 'Normalization', 'Feature Creation', 'Lag Creation']` | Order of the steps to be executed in the pipeline.                         |
-| `nan_treatment`      | `'drop'`                                                                                                                 | The treatment for missing values. Options: 'drop', 'linear interpolation'  |
+| `nan_treatment`      | `'drop'`                                                                                                                 | The treatment for missing values. Options: 'drop' (default), 'linear interpolation', 'fill linear' (alias)  |
 | `lag_train`          | `{'variable_name': 1}`                                                                                                   | The lags for each variable to be applyed during training.                      |
 | `lag_transform`      | `{'variable_name': 1}`                                                                                                   | The lags for each variable to be applyed during transformation.                |
 | `start_date`         | `'2020-01-01 00:00:00'`                                                                                                  | The start date for the dataset. Format: 'YYYY-MM-DD HH:MM:SS'                  |
@@ -58,6 +58,7 @@ Neither method mutates the passed DataFrames.
 
 - If `variable_columns` is not set, it is inferred during `fit` as all columns except the target.
 - Polynomial features are created automatically when `degree > 1`.
+- Rows with NaN in any model input get a NaN prediction (the output always has one value per input row); rows with NaN features or NaN target are left out of the fit.
 - `clipping` uses Q1/Q3 of the training target when replacing out-of-bounds predictions.
 - The wrapper enforces that `wrapper.target` is set before training or retraining; otherwise, a `ValueError` is raised to fail fast when the configuration is incomplete.
 

models/linear_regression/index.yaml

@@ -8,8 +8,8 @@ requirements:
 - cloudpickle==3.1.2
 - mlflow==3.8.1
 - git+ssh://git@github.com/Aignosi/sientia-dataops-library.git@1.11.0
-- git+ssh://git@github.com/Aignosi/sientia-model-library.git@0.14.0
+- git+ssh://git@github.com/Aignosi/sientia-model-library.git@0.14.3
-version: 1.2.0
+version: 1.2.2
 type: model
 model:
   class: LinearRegressionModel

models/linear_regression/linear_regression_model.py

@@ -64,6 +64,15 @@ class LinearRegressionModel(BaseEstimator, TransformerMixin):
         """
         return self.regr
 
+    def _check_model_features(self, columns: pd.Index, model_features: list) -> None:
+        """Raise when columns lack any feature the regressor was fitted with."""
+        missing_model_features = [c for c in model_features if c not in columns]
+        if missing_model_features:
+            raise ValueError(
+                'Prediction data is missing required transformed feature columns: '
+                + ', '.join(missing_model_features)
+            )
+
     def create_poly_features(self, input_data: pd.DataFrame, fit: bool = False) -> pd.DataFrame:
         """
         Function to create polynomial features
@@ -110,15 +119,55 @@ class LinearRegressionModel(BaseEstimator, TransformerMixin):
 
         return input_data
 
+    def _fittable_rows(
+        self, features: pd.DataFrame, target: pd.Series
+    ) -> tuple[pd.DataFrame, pd.Series]:
+        """
+        Keep only what LinearRegression can fit: inf becomes NaN, all-null columns are dropped
+        (degree 1 only) and rows with a NaN feature or a NaN target are removed.
+
+        Raises:
+            ValueError: If no rows are left.
+        """
+        # Handle infinite values
+        features = features.astype(float).replace([np.inf, -np.inf], np.nan)
+
+        # Drop columns with all null values. With degree > 1 they are kept (PolynomialFeatures
+        # needs every variable column), so the row mask below leaves no rows and fit raises.
+        if self.degree == 1:
+            all_null_cols = features.columns[features.isnull().all()].tolist()
+            features = features.drop(columns=all_null_cols)
+            if self.verbose and all_null_cols:
+                print(f'{len(all_null_cols)} columns with all null values were dropped.')
+
+        # Rows with NaN in any feature or in the target cannot be fitted (SIENTIAPDE-2215).
+        # Masked before PolynomialFeatures, which rejects NaN.
+        mask = features.notna().all(axis=1) & target.notna()
+        features = features[mask]
+        target = target[mask]
+        if features.empty:
+            raise ValueError(
+                'No training rows left after removing rows with NaN features or NaN target.'
+            )
+        if self.verbose:
+            print(f'X_train shape after removing NaN rows: {features.shape}')
+            print(f'y_train shape after removing NaN rows: {target.shape}')
+        return features, target
+
     def fit(self, input_data: pd.DataFrame) -> 'LinearRegressionModel':
         """
-        Function to fit the model
+        Function to fit the model. Rows with NaN features or a NaN target are left out of the fit.
 
         Args:
             input_data (pandas.DataFrame): The data used to fit the Linear Regression model
 
         Returns:
             LinearRegressionModel: The prediction model object
+
+        Raises:
+            ValueError: If target_variable is unset, a feature column is missing, or no
+                training rows are left after removing rows with NaN features or NaN target.
+            KeyError: If the target column is missing from input_data.
         """
         if self.verbose:
             # Display the header
@@ -153,30 +202,14 @@ class LinearRegressionModel(BaseEstimator, TransformerMixin):
             print(f'X_train shape: {X_train.shape}')
             print(f'y_train shape: {y_train.shape}')
 
+        X_train, y_train = self._fittable_rows(X_train, y_train)
+
         # Create polynomial features
         if self.degree > 1:
             X_train = self.create_poly_features(X_train, fit=True)
 
-        # Handle infinite values
+        self.q1_target = float(y_train.quantile(0.25))
-        X_train = X_train.replace([np.inf, -np.inf], np.nan)
+        self.q3_target = float(y_train.quantile(0.75))
-
-        # Drop columns with all null values
-        all_null_cols = X_train.columns[X_train.isnull().all()].tolist()
-        X_train = X_train.drop(columns=all_null_cols)
-
-        if self.verbose and all_null_cols:
-            print(f'{len(all_null_cols)} columns with all null values were dropped.')
-
-        # Get mask of rows with any NaN values
-        mask = X_train.notna().all(axis=1)
-        X_train = X_train[mask]
-        y_train = y_train[mask]
-        if self.verbose:
-            print(f'X_train shape after removing NaN rows: {X_train.shape}')
-            print(f'y_train shape after removing NaN rows: {y_train.shape}')
-
-        self.q1_target = y_train.quantile(0.25)
-        self.q3_target = y_train.quantile(0.75)
 
         # Fit the model
         self.regr.fit(X_train, y_train)
@@ -211,7 +244,7 @@ class LinearRegressionModel(BaseEstimator, TransformerMixin):
             input_data (pandas.DataFrame): The data used to predict the target variable
 
         Returns:
-            numpy.ndarray: The predicted target variable
+            numpy.ndarray: One prediction per input row; NaN where any model input is NaN
         """
         if self.verbose:
             # Display the header
@@ -228,38 +261,41 @@ class LinearRegressionModel(BaseEstimator, TransformerMixin):
                 + ', '.join(missing_features)
             )
 
-        X_test = input_data[self.variable_columns]
+        # astype(float) first: laborious sends object columns with None, and replace() on
+        # object dtype raises a pandas downcasting FutureWarning on every call.
+        X_test = input_data[self.variable_columns].astype(float).replace([np.inf, -np.inf], np.nan)
         if self.verbose:
             print(f'X_test shape: {X_test.shape}')
 
-        # Create polynomial features
+        model_features = list(self.regr.feature_names_in_)
+        # Rows with any NaN model input get a NaN prediction, so the output keeps the input
+        # length (SIENTIAPDE-2215). With degree > 1 the mask uses the raw variable columns,
+        # because PolynomialFeatures rejects NaN.
         if self.degree > 1:
-            X_test = self.create_poly_features(X_test, fit=False)
+            row_mask = X_test.notna().all(axis=1)
-        X_test = X_test.replace([np.inf, -np.inf], np.nan)
+        else:
+            self._check_model_features(X_test.columns, model_features)
+            row_mask = X_test[model_features].notna().all(axis=1)
 
-        model_features = self.regr.feature_names_in_
+        y_pred = np.full(len(input_data), np.nan)
-        # Ensure required model features exist
+        if not row_mask.any():
-        missing_model_features = [c for c in model_features if c not in X_test.columns]
+            return y_pred
-        if missing_model_features:
-            raise ValueError(
-                'Prediction data is missing required transformed feature columns: '
-                + ', '.join(missing_model_features)
-            )
 
-        # Get mask of rows with any NaN values
+        x_valid = X_test[row_mask]
-        mask = X_test.notna().all(axis=1)
+        if self.degree > 1:
-        X_test = X_test[mask]
+            x_valid = self.create_poly_features(x_valid, fit=False)
+            self._check_model_features(x_valid.columns, model_features)
         if self.verbose:
-            print(f'X_test shape after removing NaN rows: {X_test.shape}')
+            print(f'X_test shape after removing NaN rows: {x_valid.shape}')
 
-        # For simplicity and to keep behavior consistent with prior tests, drop NaN rows before prediction
+        valid_pred = self.regr.predict(x_valid[model_features])
-        y_pred = self.regr.predict(X_test[model_features])
 
         if self.clipping:
-            for i in range(len(y_pred)):
+            for i in range(len(valid_pred)):
-                if y_pred[i] > self.clipping['max']:
+                if valid_pred[i] > self.clipping['max']:
-                    y_pred[i] = self.q3_target
+                    valid_pred[i] = self.q3_target
-                elif y_pred[i] < self.clipping['min']:
+                elif valid_pred[i] < self.clipping['min']:
-                    y_pred[i] = self.q1_target
+                    valid_pred[i] = self.q1_target
 
+        y_pred[row_mask.to_numpy()] = valid_pred
         return y_pred

models/linear_regression/schemas.yaml

@@ -61,10 +61,10 @@ components:
           description: "Order of the steps to be executed in the pipeline."
         nan_treatment:
           type: string
-          enum: ["drop", "linear interpolation"]
+          enum: ["drop", "linear interpolation", "fill linear"]
           default: "drop"
           example: "drop"
-          description: "The treatment for missing values."
+          description: "The treatment for missing values. 'fill linear' is an alias of 'linear interpolation'."
         lag_train:
           type: object
           x-feature: true

models/xgboost/index.yaml

@@ -11,8 +11,8 @@ requirements:
 - rich==14.3.2
 - joblib==1.5.3
 - git+ssh://git@github.com/Aignosi/sientia-dataops-library.git@1.11.0
-- git+ssh://git@github.com/Aignosi/sientia-model-library.git@0.14.0
+- git+ssh://git@github.com/Aignosi/sientia-model-library.git@0.14.3
-version: 1.2.0
+version: 1.2.1
 type: model
 model:
   external: true

runtime/basic.yaml

@@ -8,7 +8,7 @@ libraries:
     version: 1.0.4
   - name: sientia_model
     source: Aignosi/sientia-model-library
-    version: ">=0.14.0"
+    version: ">=0.14.3"
   - name: sientia_do
     source: Aignosi/sientia-dataops-library
     version: ">=1.11.0"

runtime/index.yaml

@@ -1,10 +1,10 @@
 runtimes:
   basic:
     path: runtime/basic.yaml
-    version: 7
+    version: 8
   single:
     path: runtime/single.yaml
     version: 1
   xgboost:
     path: runtime/xgboost.yaml
-    version: 7
+    version: 8

runtime/xgboost.yaml

@@ -8,7 +8,7 @@ libraries:
     version: 1.0.4
   - name: sientia_model
     source: Aignosi/sientia-model-library
-    version: ">=0.14.0"
+    version: ">=0.14.3"
   - name: sientia_do
     source: Aignosi/sientia-dataops-library
     version: ">=1.11.0"

suse-store_DIFF.md

@@ -2,8 +2,8 @@
 
 ## Version Change
 
-- **Previous version**: 9
+- **Previous version**: 10
-- **New version**: 10
+- **New version**: 11
 
 ## Model Changes
 
@@ -14,8 +14,8 @@
 
 ### 🔄 Updated Models (2)
 
-- **linear_regression** (version: 1.2.0)
+- **linear_regression** (version: 1.2.2)
-- **xgboost** (version: 1.2.0)
+- **xgboost** (version: 1.2.1)
 
 
 ## Summary