Transformers: how do they work internally?

Table of Contents Introduction Input Embedding Positional Encoding (PE) The Encoder Self-attention Mecanism Multi-head attention mechanism Feedforward Network The Decoder Masked Multi-head Attention Multi-head Attention Feedforward Network Linear and Softmax Layer Transformer Training Conclusion Introduction The Transformer is currently one of the most popular architectures for NLP. We can periodically...

Continue reading...

Python Profiling – Memory Profiling (Part 3, Final)

Table of Contents memory_profiler PySpy DISassembling Final Recommendations memory_profiler Similar to line_profiler, memory_profiler provides detailed memory usage measurements, with the aim of efficiently reducing memory consumption and optimizing memory usage to improve application performance.. ⚠️ Before starting using this tool, it is important to mention the impact on the execution...

Continue reading...

DDD Clean Architecture Template

(GitHub Repo: https://github.com/alulema/DDD-CleanArchitectureTemplate) These last 10 months I’ve been delighted working with ASP.NET Core, considering the improvements made by Microsoft to this framework, and the option to work on my Macbook as well  However, an additional job to be done has been migrating the inherited skills and code snippets to its...

Continue reading...

C# Sudoku Solver

(GitHub Repo: https://github.com/alulema/SudokuSolverNet) I was revisiting a couple of basic AI concepts:  Depth First Search and Constraint Propagation, and I found a very good explanation by Professor Peter Norvig (Solving Every Sudoku Puzzle), I just want to add a couple of simple explanations for a better understanding of the concepts. Constraint...

Continue reading...

TensorFlow High-Level Libraries: TF Estimator

TensorFlow has several high-level libraries allowing us to reduce time modeling all with core code. TF Estimator makes it simple to create and train models for training, evaluating, predicting and exporting. TF Estimator provides 4 main functions on any kind of estimator: estimator.fit() estimator.evaluate() estimator.predict() estimator.export() All predefined estimators are...

Continue reading...

Categories