Build an AI/ML Football Analysis system with YOLO, OpenCV, and Python

Education

Introduction

In this tutorial, we will walk you through building a football analysis project utilizing state-of-the-art AI/ML techniques. We will leverage YOLO (You Only Look Once) for object detection, OpenCV for video manipulation, and perform various analytical tasks from scratch. This project will detect, track players, referees, and balls throughout the video and provide statistical insights on the game’s progression.

Table of Contents

Getting the Video

To start, you'll need a video for the analysis. We will use a dataset from Kaggle called DFL Bundesliga Data Shootout:

  • Download the video 08_f_d334.mp4.
  • Create an empty project folder named football_analysis, with subfolders: input/videos to store the downloaded video.

Introduction to Object Detection

Object detection involves recognizing and locating objects within an image. We will use YOLO, a state-of-the-art object detection model. Below is a minimal code snippet to load YOLO v8 model using ultralytics library:

[from ultralytics import YOLO](https://www.topview.ai/blog/detail/cifar-10-image-classification-using-ultralytics-yolov8)

model = YOLO('yolov8x')
results = model.predict('input/videos/08_f_d334.mp4', save=True)

Training Custom YOLO Model

We identified issues with the pre-trained YOLO model: suboptimal ball detection and unwanted detections (people off the field). To address these, we'll fine-tune YOLO using a custom dataset prepared via Roboflow.

  1. Create a Roboflow account and download the custom dataset.
  2. Here's a snippet to train YOLO with a custom dataset:
from ultralytics import YOLO

## Introduction
dataset_path = 'path/to/your/dataset'
model = YOLO('path/to/yolo/weights')
## Introduction
model.train(data=dataset_path, epochs=100, imgsz=640)
## Introduction
model.save('best.pt')

Utilizing the Custom Model

Use the custom-trained model for object detection in the video. You can save the resulting video with detected bounding boxes by running:

model = YOLO('best.pt')
results = model.predict('input/videos/08_f_d334.mp4', save=True)

Player and Ball Tracking

Tracking involves associating the same objects across different frames. We’ll use ByteTracker for efficient tracking. Here’s how to implement it:

import supervision as sv

tracker = sv.ByteTracker()
## Introduction
for frame_num, frame in enumerate(video_frames):
    results = model.predict(frame)
    tracker.update(results)

Annotating the Video

To improve the visual representation, we will replace bounding boxes with circles underneath players and triangles for the ball:

  1. Read and process the video.
  2. Draw annotations as shown in this example:
import cv2

## Introduction
for player in player_detections:
    cv2.ellipse(frame, center, axes, angle, start_angle, end_angle, color, thickness)

Assigning Players to Teams

Players will be assigned to teams based on their jersey colors by clustering image pixels:

from sklearn.cluster import KMeans

def get_team_colors(image):
    kmeans = KMeans(2)
    labels = kmeans.fit_predict(image_pixels)
    # Assign team colors based on cluster centroids
    return kmeans.cluster_centers_

Measuring Camera Movement

Camera movement can introduce errors in tracking. Using Optical Flow, we can estimate camera movement and stabilize player movements:

import cv2

## Introduction
prev_gray = cv2.cvtColor(prev_frame, cv2.COLOR_BGR2GRAY)
next_gray = cv2.cvtColor(next_frame, cv2.COLOR_BGR2GRAY)
flow = cv2.calcOpticalFlowFarneback(prev_gray, next_gray, None)

View Transformation

To convert player movements from pixels to real-world distances, perspective transformation is applied:

h, status = cv2.findHomography(src_points, dst_points)
## Introduction
transformed_points = cv2.perspectiveTransform(points, h)

Calculating Speed and Distance

Finally, compute player speed and distance covered using the transformed real-world positions:

import numpy as np

## Introduction
distance = np.linalg.norm(point1 - point2)
## Introduction
speed_kmh = (distance / time_seconds) * 3.6

Conclusion

In this project, we’ve built a comprehensive football analysis system using advanced AI/ML techniques:

  • Object detection with YOLO.
  • Custom training with annotated data.
  • Efficient tracking and visualization.
  • Assigning players to teams based on colors.
  • Camera motion estimation and stabilization.
  • Perspective transformation.
  • Calculating and visualizing speed and distance.

This robust project will significantly enhance your resume and provide real-world insights into machine learning and computer vision.

Keywords

  • Football Analysis
  • YOLO
  • Object Detection
  • Machine Learning
  • OpenCV
  • Player Tracking
  • Video Annotation
  • Optical Flow
  • Perspective Transformation
  • Speed Calculation

FAQ

Q1: What is YOLO? A1: YOLO (You Only Look Once) is a state-of-the-art object detection algorithm known for its speed and accuracy. It works by dividing the image into a grid and predicting bounding boxes and probabilities for each grid cell.

Q2: Why do we need perspective transformation? A2: Perspective transformation is necessary to convert the camera's distorted view of a football field into an accurate representation of real-world distances. This allows us to measure player movements in meters and calculate their speed correctly.

Q3: How do you handle non-detections of the football in some frames? A3: Motion interpolation is utilized to fill in the missing ball detections by guessing its position based on previous and next frames, ensuring a more continuous and accurate tracker.

Q4: How are players assigned to teams in this system? A4: Players are assigned to teams based on the colors of their jerseys. The system uses K-means clustering to group image pixels into clusters corresponding to different team colors.

Q5: What programming languages and libraries are used in this tutorial? A5: The project is implemented using Python, leveraging libraries like YOLO (via ultralytics), OpenCV for computer vision tasks, Supervision for tracking, and sKLearn for clustering.

Ready to build this football analysis system? Follow the steps in this tutorial, and you’ll have a powerful tool to analyze football matches with AI!