Your First Machine Learning Project in Python: A Step-by-Step Guide

A good first machine learning project in Python is small, honest and reproducible. Load a public dataset, split it before doing anything else, compare a baseline with two real models using cross-validation, test once on data the model has never seen, and publish the code with a clear README. This guide builds exactly that with scikit-learn 1.9.1, and every code block below was run before publishing.

At a glance

• Library: scikit-learn 1.9.1, released September 2026, which needs Python 3.11 or newer
• Dataset: Breast Cancer Wisconsin (Diagnostic), bundled with scikit-learn, so no download and no internet needed
• Size: 569 samples, 30 numeric features, 212 malignant and 357 benign
• Task: binary classification
• Models: a do-nothing baseline, logistic regression in a pipeline, and a random forest
• Metrics: recall and precision for the malignant class, ROC AUC and a confusion matrix
• Time: one weekend

Why this dataset, and what it is not

The load_breast_cancer page describes a copy of the UCI Breast Cancer Wisconsin (Diagnostic) dataset. The UCI page says the 30 features are computed from a digitised image of a fine needle aspirate of a breast mass, and the data is licensed CC BY 4.0, so credit the source in your README.

It suits a first project for three reasons. It ships with scikit-learn, so it works in a college lab with patchy internet. It is clean, so you spend your time on modelling, not on fixing broken rows. And it forces a real question: is missing a cancer worse than a false alarm? Answering that well is what separates a project from a copied notebook.

Be clear in your README that this is a learning exercise, not a medical tool.

Step 1: set up a clean environment

Following the scikit-learn install guide, create a virtual environment so the project does not clash with other Python work on your laptop. On Windows:

python -m venv ml-env
ml-env\Scripts\activate
pip install -U scikit-learn pandas
python -c "import sklearn; print(sklearn.__version__)"

On macOS or Linux, activate with source ml-env/bin/activate instead. Run the Python blocks below in order, in one Jupyter notebook or one script.

Step 2: load the data and look at it

from sklearn.datasets import load_breast_cancer

X, y = load_breast_cancer(return_X_y=True, as_frame=True)
y = (y == 0).astype(int).rename("malignant")  # flip labels so 1 means malignant

print(X.shape)               # (569, 30)
print(y.value_counts())      # 0: 357 benign, 1: 212 malignant
print(X.isna().sum().sum())  # 0 missing values
print(X.iloc[:, :4].describe().round(2))

Why flip the labels? In this dataset 0 means malignant and 1 means benign. scikit-learn's recall and precision treat 1 as the positive class by default, so without the flip your recall score would measure how well you find benign cases. Small details like this are exactly what interviewers ask about.

The describe output shows another detail: mean radius averages about 14 while mean area averages about 655. Features on such different scales confuse models like logistic regression, which is why the next steps scale them, inside a pipeline.

Step 3: split before you do anything else

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=42
)
print(len(X_train), len(X_test))  # 455 114

The scikit-learn guide to common pitfalls puts it plainly: always split into train and test first, before any preprocessing, and never call fit on the test data. stratify=y keeps the same share of malignant cases in both parts, and a fixed random_state makes your results repeatable.

From here on, the test set is locked away until Step 6.

Step 4: a baseline and two models, compared fairly

from sklearn.dummy import DummyClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import StratifiedKFold, cross_validate
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
logistic = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000))
models = {
    "baseline": DummyClassifier(strategy="most_frequent"),
    "logistic": logistic,
    "forest": RandomForestClassifier(n_estimators=300, random_state=42),
}

for name, model in models.items():
    scores = cross_validate(model, X_train, y_train, cv=cv,
                            scoring=["accuracy", "recall", "roc_auc"])
    print(f"{name:9} accuracy {scores['test_accuracy'].mean():.3f}  "
          f"recall {scores['test_recall'].mean():.3f}  "
          f"roc_auc {scores['test_roc_auc'].mean():.3f}")

On scikit-learn 1.9.1 this printed the following mean scores across five folds of the training data. Your numbers may differ slightly on other versions.

Read the baseline row twice. A model that always answers benign scores 62.6% accuracy and catches no cancers at all. That is why accuracy alone is a poor headline for this problem, and why recall belongs in your README.

The pipeline matters too. Because the scaler sits inside the pipeline, cross-validation fits it on four folds and applies it to the fifth, so no information leaks from the validation fold. Scaling the whole dataset first would quietly inflate every score.

Logistic regression wins here, and it is also the easier model to explain. Choose it.

Step 5: choose a decision threshold without touching the test set

By default, predict labels a case malignant when the predicted probability is above 0.5. scikit-learn's page on tuning the decision threshold notes that this default is rarely ideal, and uses tumour detection as its example of a case where high recall comes first. Pick a threshold from cross-validated predictions on the training data:

from sklearn.metrics import precision_score, recall_score
from sklearn.model_selection import cross_val_predict

train_prob = cross_val_predict(logistic, X_train, y_train, cv=cv,
                               method="predict_proba")[:, 1]
for threshold in [0.5, 0.4, 0.3, 0.2]:
    pred = (train_prob >= threshold).astype(int)
    print(f"threshold {threshold}: recall {recall_score(y_train, pred):.3f}  "
          f"precision {precision_score(y_train, pred):.3f}")

The run gave recall of 0.953, 0.965, 0.976 and 0.982 as the threshold fell from 0.5 to 0.2, while precision fell from 0.976 to 0.923. A threshold of 0.3 catches more malignant cases for a small rise in false alarms, so that is the choice here. Write the reasoning in your README; the trade-off is the interesting part of the project. Once you are comfortable, scikit-learn's TunedThresholdClassifierCV automates this search with its own internal cross-validation.

Step 6: test once, then stop tuning

from sklearn.metrics import classification_report, confusion_matrix, roc_auc_score

THRESHOLD = 0.3
final = logistic.fit(X_train, y_train)
test_prob = final.predict_proba(X_test)[:, 1]
test_pred = (test_prob >= THRESHOLD).astype(int)

print(confusion_matrix(y_test, test_pred))
print(classification_report(y_test, test_pred, target_names=["benign", "malignant"]))
print("ROC AUC:", round(roc_auc_score(y_test, test_prob), 3))

On the 114 test cases the confusion matrix was [[71 1] [1 41]]. Rows are the true class and columns the prediction, so the model correctly flagged 41 of the 42 malignant cases, missed one, and raised one false alarm among 72 benign cases. Test ROC AUC was 0.996. With the default 0.5 threshold, the same model missed three malignant cases.

Now stop. If you go back and tweak the model after seeing these numbers, the test set is no longer an honest estimate of performance on new data.

Step 7: save the model and the evidence

import json
from pathlib import Path

import joblib
import sklearn

Path("models").mkdir(exist_ok=True)
Path("reports").mkdir(exist_ok=True)
joblib.dump(final, "models/model.joblib")

metrics = {
    "model": "StandardScaler + LogisticRegression",
    "threshold": THRESHOLD,
    "test_recall_malignant": round(recall_score(y_test, test_pred), 3),
    "test_precision_malignant": round(precision_score(y_test, test_pred), 3),
    "test_roc_auc": round(roc_auc_score(y_test, test_prob), 3),
    "scikit_learn_version": sklearn.__version__,
}
Path("reports/metrics.json").write_text(json.dumps(metrics, indent=2))

Recording the version is not fussiness. scikit-learn's model persistence guide says there is no supported way to load a model trained with a different scikit-learn version, and warns never to load a pickle file from an untrusted source. That is also why you should not commit the model file to a public repo: anyone can rebuild it from your code in seconds.

Step 8: put it on GitHub so it counts

A tidy repository tells a recruiter more than the accuracy figure does. Use this layout:

breast-cancer-classifier/
    README.md
    requirements.txt
    .gitignore            (lists ml-env/, models/, __pycache__/, .ipynb_checkpoints/)
    notebooks/01_explore.ipynb
    src/train.py
    reports/metrics.json

Create requirements.txt with python -m pip freeze > requirements.txt, or pin the essentials by hand, starting with scikit-learn==1.9.1. Move the final code from the notebook into src/train.py so it runs end to end with one command.

Your README needs six short sections:

1. Problem: what you predict and why recall matters more than accuracy here.
2. Data: the UCI dataset name, its CC BY 4.0 licence and a link.
3. Method: split, baseline, two models, cross-validation, threshold choice.
4. Results: the cross-validation table and the test confusion matrix.
5. How to run: three commands from clone to results.
6. Limits and next steps: a learning project, not a diagnostic tool, and what you would try next.

Commit in small steps with clear messages. A history that shows exploration, then modelling, then clean-up reads as your own work. If you are a student, our guide to the GitHub Student Developer Pack explains the free tools available to you.

Where to go next

• Try a stronger model. Add HistGradientBoostingClassifier to the comparison in Step 4 and see whether it beats logistic regression under the same cross-validation.
• Use Indian data. Repeat the whole method on a dataset you care about. Our list of final-year project ideas with AI and data has starting points.
• Strengthen your Python. If the code above felt hard to read, our PCEP exam guide for India covers the fundamentals that make machine learning code easier.

How CertTulen Academy can help

If you want a structured path, CertTulen Academy's Python Entry-Level Programming (PCEP) course builds the Python fundamentals this project relies on, in live, instructor-led online classes. When you are ready to run models in production, the AI-300 Machine Learning Operations Engineer course covers pipelines, deployment and monitoring with hands-on labs. Course pages show prices in INR with UPI, card and netbanking checkout. Use our contact page or WhatsApp +91 93618 83583 to ask for a batch that suits your time zone.