Category: Machine Learning

🚀 The Evolution of YOLO 🚀

The YOLO (You Only Look Once) series is a real-time object detection algorithm that uses convolutional neural network (CNN). It has dramatically shaped the landscape of real-time computer vision. Each iteration of YOLO brings something unique to the table, enhancing the capabilities and applications of object detection. Let’s dive into the evolution of Yolo, details of each major YOLO model, the companies and organizations behind them, and how they contribute to the evolution of AI.

🌐 Timeline of Key YOLO Models and Their Innovators:

1️⃣ Joseph Redmon 🧠:

  • YOLO V1 (2016): Redmon introduced the first YOLO model, which revolutionized object detection by framing it as a single regression problem instead of a classification task. This approach allowed YOLO to detect objects in images at unprecedented speeds, making it suitable for real-time applications.
  • YOLO V2 & YOLO V3: These versions refined the detection process, improving accuracy with techniques like anchor boxes and multi-scale predictions. YOLO V3 was especially known for its balance between speed and accuracy, making it a benchmark for real-time object detection.

2️⃣ AlexeyAB & WongKinYiu 🔧:

  • YOLO V4: Building on Redmon’s work, AlexeyAB introduced YOLO V4, which incorporated advanced techniques like CSPDarknet53 as the backbone, PANet for path aggregation, and various other improvements that significantly boosted detection accuracy while maintaining speed.
  • Scaled YOLO V4: WongKinYiu extended YOLO V4 by introducing scaling capabilities, allowing the model to adapt to different sizes depending on the computational resources available.
  • YOLO-R & YOLO V7: These versions continued to refine the architecture, focusing on edge-device efficiency without compromising accuracy, and further solidifying YOLO’s role in lightweight, real-time applications.

3️⃣ Ultralytics (Glenn Jocher) 💻:

  • YOLO V5: Ultralytics’ YOLO V5 made a significant impact by offering an easier-to-use version of YOLO with extensive support for PyTorch. It’s known for its ease of training, deployment, and integration into various projects. YOLO V5 became the go-to model for many practitioners due to its flexibility and performance.
  • YOLO V8: The latest from Ultralytics, YOLO V8, pushes the envelope with state-of-the-art performance, integrating the latest research advancements, and focusing on deployment efficiency in various environments, from cloud to edge devices.

4️⃣ Meituan Technical Team 🌟:

  • YOLO V6: Aimed at balancing speed and accuracy, YOLO V6 from Meituan was developed with a focus on real-world applications where inference speed on edge devices is critical. It leverages modern techniques like EfficientNet as the backbone to optimize performance.
  • YOLO V6 3.0: This update introduced further refinements in the model architecture, allowing it to perform even better on resource-constrained devices, making it ideal for mobile and embedded applications.

5️⃣ Baidu 🧬:

  • PP-YOLO Series: Baidu’s PP-YOLO and its successors (V2 and beyond) are optimized for PaddlePaddle, an AI framework developed by Baidu. PP-YOLO models integrate many of the latest research advancements in object detection, providing a powerful tool for various commercial applications. Baidu’s focus on enhancing speed and accuracy makes PP-YOLO particularly well-suited for industrial AI applications where efficiency is key.

6️⃣ Megvii Technology 🏢:

  • YOLOX: Megvii’s YOLOX introduces a new paradigm by decoupling the head of the network into classification and regression branches, improving performance and making it easier to adapt to different tasks. It’s optimized for versatility, robustness, and ease of deployment, particularly in scenarios requiring high accuracy and low latency. YOLOX’s advancements make it a strong contender in the commercial AI space.

7️⃣ Alibaba DAMO Academy 🏛️:

  • DAMO YOLO: Alibaba’s DAMO Academy has taken YOLO to new heights with its DAMO YOLO models, which focus on specialized applications requiring high precision. These models leverage PyTorch and Apache licensing and are designed for integration into Alibaba’s vast ecosystem, ensuring scalability and robustness in production environments.

8️⃣ Deci AI 🛠️:

  • YOLO-NAS: Deci AI introduces a novel approach with YOLO-NAS, utilizing Neural Architecture Search (NAS) to automatically optimize YOLO models for specific tasks. This results in highly efficient, custom-tailored models that excel in specific applications, providing a significant edge in performance and deployment flexibility.
History of Yolo computer vision model development

Image Credit : OpenCV.ai

📜 Licensing Overview:

  • MIT License: Used in the early YOLO versions, allowing for broad use and modification with few restrictions.
  • GPL License: Encourages collaboration while ensuring that derivative work remains open source.
  • Apache License: Offers a balance between open-source freedom and commercial use, widely adopted in enterprise environments.

💡 Frameworks:

  • Darknet: The original framework used in early YOLO versions, known for its speed and efficiency.
  • PyTorch: Dominates recent YOLO models, providing flexibility and a rich ecosystem for development and deployment.
  • PaddlePaddle: Baidu’s in-house framework, optimized for PP-YOLO models, ensuring tight integration with Baidu’s AI infrastructure.

As we continue to push the boundaries of AI, the evolution of YOLO to what it is today has been of immense help to computer vision developers. Each new iteration not only refines performance but also expands the possibilities of what AI can achieve in real-world applications. Whether you’re developing on the edge or deploying in the cloud, the YOLO family offers a model for every need. 🌍

Basic Machine Learning Optimization Algorithms

Keeping up with my tradition of posting my old handwritten notes, here are my notes on basic Machine Learning optimization algorithms. Optimization algos in ML help minimize the cost function, thereby helping to reduce the error between predicted value and actual value

1) Most popular – Gradient Descent

Gradient descent equation

Gradient Descent is used in linear regression, logistic regression & early implementation of neural networks

Gradient Descent
Gradient Descent ML algorithm

2) ADAM = ADAptive Moment estimation

Adam optimization algorithm does not use single global learning rate , but a different learning rate for every single parameter

Adam Optimization ML algorithm
Adam Optimization ML algorithm

I hope you liked my notes on basic machine learning optimization algorithms. Feel free to comment here or on my Linkedin post.

Hand written notes on Neural Networks and ML course by Andrew Ng

About 2018 when I started working on Machine learning I took many courses. Here are my hand written notes on Neural Networks and ML course by Andrew Ng. It focuses on the fundamental concepts covered in the course, including Logistic Regression, Neural Networks, and Softmax Regression. Buckle up for some equations and diagrams!

Part 1: Logistic Regression – The Binary Classification Workhorse

Logistic regression reigns supreme for tasks where the target variable (y) can only take on two distinct values, typically denoted as 0 or 1. It essentially calculates the probability (a) of y belonging to class 1, given a set of input features (x). Here’s a breakdown of the process:

  1. Linear Combination: The model calculates a linear score (z) by taking a weighted sum of the input features (x) and their corresponding weights (w). We can represent this mathematically as:

    z = w_1x_1 + w_2x_2 + … + w_nx_n

    (where n is the number of features)
  2. Sigmoid Function: This linear score (z) doesn’t directly translate to a probability. The sigmoid function (σ) steps in to transform this score into a value between 0 and 1, representing the probability (a) of y belonging to class 1. The sigmoid function is typically defined as:

a = \sigma(z) = \frac{1}{1 + e^{-z}}

Sigmoid Function Plot / Logistic Curve


Key takeaway: 1 – a represents the probability of y belonging to class 0. This is because the sum of probabilities for both classes must always equal 1.

Part 2: Demystifying Neural Networks – Building Blocks and Forward Propagation

  1. Perceptrons – The Basic Unit: Neural networks are built using perceptrons, the fundamental unit inspired by biological neurons. A perceptron takes weighted inputs (just like logistic regression), performs a linear transformation, and applies an activation function to generate an output.
  2. Activation Functions: While sigmoid functions are common in logistic regression and the initial layers of neural networks, other activation functions like ReLU (Rectified Linear Unit) can also be employed. These functions introduce non-linearity, allowing the network to learn more complex patterns in the data.
  3. Layering Perceptrons: Neural networks are not limited to single perceptrons. We can stack multiple perceptrons into layers, where each neuron in a layer receives outputs from all the neurons in the previous layer. This creates a complex network of interconnected units.
  4. Forward Propagation: Information flows through the network in a forward direction, layer by layer. In each layer, the weighted sum of the previous layer’s outputs is calculated and passed through an activation function. This process continues until the final output layer produces the network’s prediction.

Part 3: Unveiling Backpropagation – The Learning Algorithm

But how do these neural networks actually learn? Backpropagation is the hero behind the scenes! It allows the network to adjust its weights and biases in an iterative manner to minimize the error between the predicted and actual outputs.

  1. Cost Function: We define a cost function that measures how well the network’s predictions align with the actual labels. A common cost function for classification problems is the cross-entropy loss.
  2. Error Calculation: Backpropagation calculates the error (difference between prediction and actual value) at the output layer and propagates it backward through the network.
  3. Weight and Bias Updates: Based on the calculated errors, the weights and biases of each neuron are adjusted in a way that minimizes the overall cost function. This process is repeated iteratively over multiple training epochs until the network converges to a minimum error state.

Part 4: Softmax Regression – Expanding Logistic Regression for Multi-Class Classification

Logistic regression excels in binary classification, but what happens when we have more than two possible class labels for the target variable (y)? Softmax regression emerges as a powerful solution!

  1. Generalizing Logistic Regression: Softmax regression can be viewed as an extension of logistic regression for multi-class problems. It calculates a set of class scores (z_i) for each possible class (i).
  2. The Softmax Function: Similar to the sigmoid function, softmax takes these class scores (z_i) and transforms them into class probabilities (a_i) using the following formula:

a_i = \frac{e^{z_i}}{\sum\limits_{j=1}^{C} e^{z_j}}

(where Σ represents the sum over all possible classes j)
Key takeaway: This function ensures that all the class probabilities (a_i) sum up to 1, which is a crucial requirement for a valid probability distribution. Intuitively, for a given input (x), only one class can be true, and the softmax function effectively distributes the probability mass across all classes based on their corresponding z_i scores.

Softmax Function Curve

  1. Interpretation of Class Probabilities: Each class probability (a_i) represents the model’s estimated probability of the target variable (y) belonging to class i, given the input features (x). This probabilistic interpretation empowers us to not only predict the most likely class but also gauge the model’s confidence in that prediction.

Part 5: Putting It All Together – Training and Cost Function for Softmax Regression

Part 5: Putting It All Together – Training and Cost Function for Softmax Regression

While we’ve focused on the mechanics of softmax, training a softmax regression model involves a cost function. Here’s a brief overview:

  1. Negative Log-Likelihood Cost Function: Softmax regression typically employs the negative log-likelihood cost function. This function penalizes the model for assigning low probabilities to the correct class and vice versa. Mathematically, the cost function can be represented as:Cost=−Σ(yi∗log(ai))
    (where y_i is 1 for the correct class and 0 otherwise)
  2. Model Optimization: During training, the model aims to minimize this cost function by adjusting its weights and biases through backpropagation. As the cost function decreases, the model learns to produce class probabilities that better reflect the underlying data distribution.

Conclusion: A Stepping Stone to Deep Learning

These blog and hand written notes on Neural Networks and ML has provided a condensed yet detailed exploration of logistic regression, neural networks, and softmax regression, concepts covered in Andrew Ng’s Advanced Learning Algorithms course. Understanding these fundamental building blocks equips you to delve deeper into the fascinating world of Deep Learning and explore more advanced architectures like Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs).Remember, this is just the beginning of your Deep Learning journey!

I hope these detailed hand written notes on Neural Networks and ML with diagrams prove helpful for your Deep Learning studies!

Rust Programming Language learning roadmap

Rust is a multi-paradigm, general-purpose programming language exploding in popularity. But what makes it special? Rust offers a unique blend of blazing speed, unparalleled memory safety, and powerful abstractions, making it ideal for building high-performance, reliable systems. This blog delves into the Rust Programming Language learning roadmap

Why Embrace Rust?

  • Unmatched Performance: Rust eliminates the need for a garbage collector, resulting in lightning-fast execution and minimal memory overhead. This makes it perfect for resource-constrained environments and applications demanding real-time responsiveness.
  • Rock-Solid Memory Safety: Rust enforces memory safety at compile time through its ownership system. This eliminates entire classes of memory-related bugs like dangling pointers and use-after-free errors, leading to more stable and secure software.
  • Zero-Cost Abstractions: Unlike some languages where abstractions incur performance penalties, Rust achieves powerful abstractions without sacrificing speed. This allows you to write expressive, concise code while maintaining peak performance.

Language Fundamentals: Understanding the Building Blocks

Syntax and Semantics: Rust borrows inspiration from C-like languages in its syntax, making it familiar to programmers from that background. However, Rust’s semantics are distinct, emphasizing memory safety through ownership and immutability by default.

Constructs and Data Structures: Rust offers a rich set of control flow constructs like if, else, loop, and while for building program logic. Data structures encompass primitive types like integers, booleans, and floating-point numbers, along with powerful composite types like arrays, vectors, structs, and enums.

Ownership System: The Heart of Rust

The ownership system is the cornerstone of Rust’s memory safety. Let’s delve deeper:

  • Ownership Rules: Every value in Rust has a single owner – the variable that binds it. When the variable goes out of scope, the value is automatically dropped, freeing the associated memory. This ensures memory is never left dangling or leaked.
  • Borrowing: Borrowing allows temporary access to a value without taking ownership. References (&) and mutable references (&mut) are used for borrowing. The borrow checker, a powerful Rust feature, enforces strict rules to prevent data races and ensure references always point to valid data.
  • Stack vs. Heap: Understanding these memory regions is crucial in Rust. The stack is a fixed-size memory area used for local variables and function calls. It’s fast but short-lived. The heap is a dynamically allocated memory region for larger data structures. Ownership dictates where data resides: stack for small, short-lived data, and heap for larger, long-lived data.

Rust programming language learning roadmap

Beyond the Basics: Advanced Features

  • Error Handling: Rust adopts an Result type for error handling. It represents either successful computation with a value or an error with an error code. This promotes explicit error handling, leading to more robust code.
  • Modules and Crates: Rust promotes code organization through modules and crates. Modules group related code within a source file, while crates are reusable libraries published on https://crates.io/.
  • Concurrency and Parallelism: Rust provides mechanisms for writing concurrent and parallel programs. Channels and mutexes enable safe communication and synchronization between threads, allowing efficient utilization of multi-core processors.
  • Traits and Generics: Traits define shared behaviors for different types, promoting code reusability. Generics allow writing functions and data structures that work with various types, enhancing code flexibility.
  • Lifetimes and Borrow Checker: Lifetimes specify the lifetime of references in Rust. The borrow checker enforces rules ensuring references are valid for their intended usage duration. This prevents data races and memory unsafety issues.

Rust’s Reach: Applications Across Domains

  • Web Development: Frameworks like Rocket and Actix utilize Rust’s speed and safety for building high-performance web services and APIs.
  • Asynchronous Programming: Async/await syntax allows writing non-blocking, concurrent code, making Rust perfect for building scalable network applications.
  • Networking: Libraries like Tokio provide efficient tools for building networking applications requiring low latency and high throughput.
  • Serialization and Deserialization: Rust’s data structures map well to various data formats like JSON and CBOR, making it suitable for data exchange tasks.
  • Databases: Several database libraries like Diesel offer safe and performant database access from Rust applications.
  • Cryptography: Rust’s strong typing and memory safety make it ideal for building secure cryptographic systems.
  • Game Development: Game engines like Amethyst leverage Rust’s performance and safety for creating high-fidelity games.
  • Embedded Systems: Rust’s resource-efficiency and deterministic memory management make it a compelling choice for resource-constrained embedded systems.

Image Credit : roadmap.sh

What is Null and Alternative Hypothesis


Null and Alternative Hypothesis is used extensively in Machine Learning. Before we answer what is null and alternative hypothesis, let us understand what is Hypothesis Testing.

Hypothesis Testing is used to assess if the difference between samples taken from population are representative of actual difference between populations themselves. 

Now why do we even conduct hypothesis testing? Suppose we are comparing the efficacy between two different exercises on 10 patients who underwent the same kind and complexity of knee replacement surgery. This should not be too difficult to do with real data. E.g. 5 patients are asked to do exercise1 and other 5 are asked to do exercise 2,  15 mins a day for 1 month after surgery. After a month, they are tested for the angle at which they can bend their knee. This comparison between patients is not difficult to make.

Now let us imagine if the same comparison is between two groups of patients from two different hospitals. The comparison quickly becomes unwieldy and introduces multiple random events which can easily affect the data. E.g. some patients do exercise after a shower, or after food, or some patients were on different medication which affected their musculoskeletal system etc. Now imagine if the comparison is not across these two groups, but across the population of the whole state of California. Here it is extremely difficult, if not impossible, to compare every single patient in the state of California. Lets park this thought for a second.

Now what is Null Hypothesis (H0):

Null hypothesis H0 is also described as “no difference” hypothesis. i.e. There is no difference between sample sets from different populations. Here we mostly see an equality relationship.

So for the example above about sample of patients from state of California, we start by assuming that null hypothesis H0 is true. i.e. all the samples from different population set are same and that there is no difference between them. We then calculate the probability of such an event (occurrence of null-hypothesis) using the a number between 0 and 1 called as p-value. Generally a value less than 0.05 is considered low probability and therefore we say that chance of having null-hypothesis is very less. Thus we reject null-hypothesis in that scenario. For a p-value greater than 0.05 we fail to reject null-hypothesis (Basically a roundabout way of saying that null-hypothesis is true). For p<0.05, when null-hypothesis is rejected, an alternative hypothesis must be adopted in its place.

What is Alternative Hypothesis (Ha) :

Alternative hypothesis is a scenario in which if you have a new claim against the default hypothesis using current data and therefore it is sort of a new news and requires data to back up this claim. You basically say that you disagree with the data at hand and that there is something new happening. The alternative hypothesis does not have a statement of equality and also uses data from pre-established success rate given in the problem statement.

In the above example of efficacy of exercise comparison for patients, a p-value < 0.05 will reject null-hypothesis and thus an alternative hypothesis takes place. From hereon, we get an opportunity to dig more deeper into the available data and see how the sample sets are different ( Remember, if null-hypothesis was true then we would have concluded that sample data are all the same and that both exercises help patients equally. Only after p-value being less than 0.05 we rejected null-hypothesis ) . We can use various statistical calculations e.g. mean angle of knee bent amongst people of exercise1 set vs mean angle of knee bent amongst people of exercise2 set. This may help us determine which exercise turned out to be better.

Patient number 2 doing exercise number 2 after knee replacement surgery. Example for what is null and alternative hypothesis


Why do we use null-hypothesis? 

Null-hypothesis seems to be an extremely simple way to start. And that is exactly what we want in statistical inference. A starting point. Null hypothesis provides an easy starting point. It is easy to describe our expectation from data when null hypothesis exist. An then by using p-value we land on to alternative hypothesis, if it is available or inferred using the data. Null and alternative hypothesis are mutually exclusive.

Bibliography:

Supervised Machine Learning for Beginners

Welcome. You are in the right place if you are just starting your journey learning Machine Learning. I found my very old notes / cheat sheet about Supervised Machine Learning for beginners when I started learning ML a long time ago. Here is the link to a high resolution pdf if you are interested.

A little primer to Supervised Machine Learning follows. Read my attached original notes for details

Supervised Learning is a field of Machine Learning that learns from examples / known outputs y and is then able to predict y from other newer values of x
Linear Regression Classification
Algo predicts an output value y from infinite possible values for a given input xDefinition : Algo predicts finite outputs / categories for a given input
Model function : 
f(x) = wx +b
Is basically a function for straight line		Model function for Logistic Regression : 
f(x) = g(z) = 1/ (1 + e-z )
Is a function describing a sigmoid 
Graph of the function
Graph of Sigmoid function

Convex plot of Cost function
Supervised Machine Learning handwritten cheat sheet

Now that you have notes on Supervised Machine Learning for beginners, other foundational ML topic you may be interested in is Null and Alternate Hypothesis. Happy reading !