Language Detector in ML.NET

Table of Contents

1. Challenge

   Step 1: Explore the ML.NET workflow

   Picture a messaging app whose users write in dozens of languages. Before it can translate a message or route it to the right support team, the app needs to know which language the message is in, and how sure it is about that guess. You'll build that piece here: a language detector that scores any sentence against English, Spanish, French, and German, learned from labeled examples instead of hand-written rules.

   Along the way, you'll describe a CSV dataset to ML.NET's Model Builder, let AutoML train and compare candidate models, and turn the generated prediction code into a console app that prints a ranked score for every language. You'll also fix a weak spot by adding examples and retraining with the trainer AutoML chose. To begin, you'll look at the tool that drives training and the dataset it learns from, so every later step has a clear place in the workflow.

   The application directory already contains the labeled dataset and a training configuration with the column settings for you to complete.

   info> Feeling stuck? Check out the matching solution/stepN/ folder for the step you're on to see a working implementation. Give it a try on your own first. The solution folder is your safety net, not your starting point. ### The ML.NET workflow and its CLI

   Every ML.NET model follows the same three-part workflow: a labeled dataset goes into training, training produces a model file, and the model scores new input. Model Builder is the ML.NET tool that automates the training part, and the ML.NET CLI, mlnet, is its command-line form. It uses AutoML, which trains and scores many candidate models for you and keeps the best one, so you don't write training code yourself. Model Builder and the CLI both read and write an .mbconfig training configuration file, and the CLI's train command runs a saved configuration.

   The CLI expects a command, so running it with only --help prints Required command was not provided. above the help text. That message is expected; the command list below it is what you're looking for. ### The labeled dataset

   The detector learns from langs.csv, where each row pairs a sentence with the language it's written in. The Id column numbers the rows, the Language column holds the label (en, es, fr, or de), and the Text column holds the sentence. The sentences are short, everyday phrases, such as greetings, directions, and questions about food or the weather.

2. Challenge

   Step 2: Configure the training dataset

   With the CLI's commands and the dataset's columns in view, you're ready to tell AutoML how to read each column. You'll edit the training configuration, a JSON file that records what each CSV column means and that the CLI reads when you train. A column's purpose is one of three roles: the label is the value the model predicts, a feature is an input the model learns from, and an ignored column is left out of training entirely. By the end of this step, the configuration tells AutoML exactly what to predict, what to learn from, and what to skip. ### The label column

   In LanguageDetector.mbconfig, the DataSource section has a ColumnProperties array with one entry per CSV column. Each entry names its column in ColumnName and sets its role in ColumnPurpose. The label is the value you want the model to predict, which for this detector is the language of each sentence. Right now, each column has a placeholder purpose for you to correct. ### The feature column

   A feature is an input the model learns patterns from. For a language detector, that input is the sentence itself. AutoML turns the text into numbers that capture its words and letter combinations, then learns which patterns go with which language. ### Columns to ignore

   Not every column carries useful information. The Id column only numbers the rows, so a model that learned from it could pick up patterns that have nothing to do with language, such as which row numbers happen to hold French sentences. Setting a column's purpose to Ignore leaves it out of training entirely.

3. Challenge

   Step 3: Train and compare models

   With every column's purpose set, the training configuration is ready to run. You'll train the detector with AutoML, which tries many trainer and parameter combinations within a short training time, and then read back the best candidates it saved. A trainer is the learning algorithm that fits the model, and a trainer family is a group of related algorithms. Each explored model gets a score: its accuracy on sentences held back from training, where 1.0 means every held-back sentence was classified correctly. By the end of this step, you'll know which trainer family produced your model. ### How AutoML searches

   mlnet train reads the training configuration, loads langs.csv, and runs an AutoML experiment for the training time the configuration sets, about 20 seconds here. AutoML tries each trainer family early in the search, then spends most of its time tuning the families that score well, and it prints one line per model with the trainer name and its MacroAccuracy score. When time runs out, it generates a console project with prediction and training code for the best model and saves the model itself as LanguageDetector.mlnet. ### The saved candidates

   After training, the CLI writes the five best trials back into LanguageDetector.mbconfig, each with a TrainerName and a Score. The first trial listed is the model AutoML selected and generated code for. Because AutoML spends most of its search refining the strongest family, these candidates usually share one trainer family, and often the same score. To see how the other families compared, scroll up through the trial lines from your training run, where they appear with lower scores.

4. Challenge

   Step 4: Predict languages from a console app

   With a trained model and its generated prediction code in place, you can turn the generated console program into a detector you call from the terminal. As generated, the program scores one fixed sample sentence and then waits for a key press. You'll make it score the sentence you pass on the command line, remove the output that describes the fixed sample, and print every score as a percentage. The result is a ranked list of likely languages for any message, which is what the messaging app needs instead of a single guess. ### The generated console program

   mlnet train generated Program.cs with a classic Main(string[] args) method. It builds a ModelInput whose Text property holds a hard-coded sentence, the first row of langs.csv, and passes it to the generated PredictAllLabels method, which returns every language with its score, sorted from highest to lowest. The args parameter holds the command-line arguments, and args[0] is the first one.

   warning> Don't run mlnet train again for the rest of the lab. It regenerates Program.cs and the other generated files every time it runs, which would replace your edits. ### Output that describes the fixed sample

   Below the ModelInput, the generated program prints three lines about its fixed sample: a banner saying it's comparing the actual language with the predicted one, a Language: line with the sample's label, and a Text: line with the sample sentence. Now that the program scores your sentence, all three describe the wrong input. You'll remove them one at a time, starting with the banner. ### The hard-coded language

   The Language: line prints fr, the label of the dataset's first row, no matter which sentence you pass in. For a German sentence, it would claim the actual language is French, right above scores that say otherwise. ### The hard-coded sentence

   The Text: line repeats the fixed sample sentence, Avez-vous une table pour deux, rather than the sentence you passed in. Since you type the sentence yourself on the command line, the output doesn't need to echo it. ### Waiting for a key press

   At the end of Main, the generated program calls Console.ReadKey(), which pauses until someone presses a key. That keeps a console window open when the program is launched by double-clicking it, but it stops a command-line tool from exiting on its own. It also throws an exception when input is redirected, such as when a script runs the program. ### The leftover prompt

   The line just above the call you removed still prints =============== End of process, hit any key to finish ===============. Without ReadKey(), nothing waits for that key, so the prompt is now misleading. ### Percentages and culture-independent formatting

   Each score is a float between 0 and 1, and the generated loop prints it as a raw number. The P standard numeric format multiplies a number by 100 and adds a percent sign, and a digit after it sets the number of decimal places. Formatting also depends on the current culture, which decides things such as whether the decimal separator is a period or a comma, so passing CultureInfo.InvariantCulture makes the output look the same on every machine. CultureInfo lives in the System.Globalization namespace.

   var label = ratio.ToString("P2", CultureInfo.InvariantCulture);
   ``` ### Passing arguments to dotnet run
   
   `dotnet run` builds and starts the project in the current folder. Everything after `--` goes to your program instead of to the `dotnet` CLI, so the quoted sentence arrives in `args[0]`. The first run after an edit also rebuilds the project, so it takes a few seconds longer.

5. Challenge

   Step 5: Extend the dataset and retrain

   Your detector scores any sentence from the command line, which also makes it easy to probe where the model is weak. A model is only as good as its examples, so you'll score a sentence the model handles poorly, add labeled sentences that cover that weak spot, and retrain. Rather than running a new AutoML search that might pick a different trainer, you'll retrain with the pipeline the CLI generated, which keeps the trainer AutoML already selected. By the end of this step, the same sentence will be recognized correctly. ### An under-represented language

   The dataset is uneven: it has 60 sentences each for English, Spanish, and French, but only 12 for German. With so few German examples, the model hasn't seen enough German words to recognize a German sentence that shares little with them. The sentence in this task, Können Sie mir bitte helfen (Can you please help me?), is one of those, so expect the model to get it wrong. ### Adding labeled examples

   The fix for an under-represented label is more labeled examples of it. The rows below are 40 everyday German sentences, each labeled de, with Id values that continue from 192, the last row of the dataset. The file uses plain comma-separated values with no quoting, so each row needs exactly three fields, and none of these sentences contains a comma. ### The generated training pipeline

   mlnet train also generated LanguageDetector.training.cs, which holds the exact pipeline AutoML selected, including its trainer and settings. The generated LanguageDetector class exposes that pipeline through a static Train method whose first parameter is the path to save the model to, and whose training-data parameter already defaults to langs.csv. To add a retrain mode, Main checks its first argument before treating it as a sentence, and a return ends Main right after training.

   if (option == "--flag")
   {
       Worker.Run("output.bin");
       return;
   }
   ``` ### Retraining from the command line
   
   Running the program with `--retrain` takes your new branch: it trains the generated pipeline on the current `langs.csv` and overwrites `LanguageDetector.mlnet`. It doesn't run an AutoML search, so it finishes in a few seconds. The branch doesn't print anything, so the only sign that it's done is the prompt coming back. ### Reading the pipeline's trainer
   
   The pipeline in `LanguageDetector.training.cs` appends one trainer from `mlContext.MulticlassClassification.Trainers`. Its name maps to the `TrainerName` values from Task 3.2:
   
   - A name ending in `Multi` is a multiclass trainer used directly, such as `LbfgsMaximumEntropy` for `LbfgsMaximumEntropyMulti`.
   - A name ending in `Ova` wraps a binary trainer in `OneVersusAll`, which trains one yes-or-no model per language, such as `OneVersusAll` around `LbfgsLogisticRegression` for `LbfgsLogisticRegressionOva`.
   
   Because retraining reuses this line, the retrained model has the same trainer AutoML selected. ### Measuring the improvement
   
   Your retrained model has seen 52 German sentences instead of 12. Scoring the same sentence again shows whether the new examples fixed the weak spot. Congratulations on completing the lab.
   
   You built a language detector in ML.NET. You described a labeled CSV to Model Builder, trained and compared candidate models with AutoML, and turned the generated code into a console app that prints a ranked percentage score for every language. Your last run showed `Können Sie mir bitte helfen` move from a wrong answer to `de` at the top.
   
   Along the way, you practiced the full ML.NET workflow from data to predictions. You also saw how growing a dataset and retraining with a fixed pipeline can improve a weak label without changing the trainer AutoML chose.