# Fichier: python_cheats/cheatsheets/performance.txt
# Cheatsheet Performance et Optimisation Python - Guide Complet


1. **Introduction** - Règles d'or et types de problèmes
2. **Mesure du temps** - time, timeit, décorateurs, contextes
3. **timeit** - Benchmarking précis avec exemples
4. **cProfile** - Profiling détaillé avec analyse
5. **line_profiler** - Profiling ligne par ligne
6. **memory_profiler** - Profiling mémoire
7. **Optimisations générales** - 8 techniques fondamentales
8. **Structures de données** - collections, array, bisect, heapq
9. **__slots__** - Optimisation classes
10. **functools** - lru_cache, cached_property, partial
11. **itertools** - Itération efficace
12. **NumPy** - Calcul vectorisé (10-100x plus rapide)
13. **Numba** - JIT compilation
14. **Cython** - Compilation C
15. **Multiprocessing** - Parallélisme CPU
16. **concurrent.futures** - Threading/Processing simplifié
17. **asyncio** - Programmation asynchrone I/O
18. **Optimisations avancées** - 8 techniques expertes
19. **Profiling tools avancés** - py-spy, scalene, viztracer, memray
20. **Optimisation database** - Batch, indexing, pooling
21. **Bonnes pratiques** - 10 règles essentielles
22. **Checklist optimisation** - Guide étape par étape
23. **Anti-patterns** - 10 erreurs communes à éviter
24. **Comparaisons détaillées** - Benchmarks concrets
25. **Profiling exemple complet** - Optimisation progressive
26. **Optimisation par cas d'usage** - 5 cas réels
27. **Monitoring production** - Outils et techniques
28. **Tips & Tricks** - 10 astuces avancées
29. **Ressources** - Outils et documentation
30. **Récapitulatif** - Golden rules et quick wins

La cheatsheet est maintenant 10x plus complète avec des exemples pratiques, benchmarks réels et cas d'usage concrets ! [RAPIDE]


[OK] INTRODUCTION À LA PERFORMANCE

# Règles d'or de l'optimisation:
# 1. "Premature optimization is the root of all evil" - Donald Knuth
# 2. Mesurer AVANT d'optimiser (profiling)
# 3. Optimiser les bottlenecks identifiés, pas tout
# 4. Maintenir la lisibilité du code
# 5. Benchmarker après chaque optimisation

# Types de problèmes de performance:
# - CPU-bound: Calculs intensifs (multiprocessing, Numba, Cython)
# - I/O-bound: Lecture/écriture (asyncio, threading)
# - Memory-bound: Utilisation mémoire (générateurs, numpy arrays)
# - Network-bound: Requêtes réseau (asyncio, connection pooling)


[OK] MESURER LE TEMPS D'EXÉCUTION

# === time.time() - Simple et direct ===
import time

start = time.time()
# Code à mesurer
result = sum(range(1000000))
end = time.time()
print(f"Temps: {end - start:.4f}s")

# time.perf_counter() - Plus précis
start = time.perf_counter()
# Code
end = time.perf_counter()
print(f"Temps: {end - start:.6f}s")

# time.process_time() - Temps CPU seulement (sans I/O)
start = time.process_time()
# Code
end = time.process_time()
print(f"Temps CPU: {end - start:.6f}s")

# === Décorateur de timing ===
import functools

def timer(func):
    """Décorateur pour mesurer temps d'exécution"""
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        result = func(*args, **kwargs)
        end = time.perf_counter()
        print(f"{func.__name__} exécuté en {end - start:.6f}s")
        return result
    return wrapper

@timer
def my_function():
    return sum(range(1000000))

# === Contexte manager pour timing ===
from contextlib import contextmanager

@contextmanager
def timer_context(name="Block"):
    """Context manager pour mesurer temps"""
    start = time.perf_counter()
    yield
    end = time.perf_counter()
    print(f"{name}: {end - start:.6f}s")

# Utilisation
with timer_context("Calcul sum"):
    result = sum(range(1000000))

# === Classe Timer réutilisable ===
class Timer:
    """Timer avec API pratique"""
    def __init__(self, name=None):
        self.name = name
        self.start_time = None
        self.elapsed = 0
    
    def __enter__(self):
        self.start()
        return self
    
    def __exit__(self, *args):
        self.stop()
        print(self)
    
    def start(self):
        """Démarre timer"""
        self.start_time = time.perf_counter()
    
    def stop(self):
        """Arrête timer"""
        if self.start_time:
            self.elapsed = time.perf_counter() - self.start_time
            self.start_time = None
    
    def __str__(self):
        name = self.name or "Timer"
        return f"{name}: {self.elapsed:.6f}s"

# Utilisation
with Timer("Mon calcul"):
    time.sleep(0.5)

timer = Timer()
timer.start()
# Code
timer.stop()
print(timer)


[OK] TIMEIT - BENCHMARKING PRÉCIS

# === Usage basique ===
import timeit

# Mesurer une expression
time_taken = timeit.timeit('"-".join(str(n) for n in range(100))', number=10000)
print(f"Temps: {time_taken:.6f}s pour 10000 exécutions")

# Mesurer avec setup
setup = "from math import sqrt"
code = "sqrt(144)"
time_taken = timeit.timeit(code, setup=setup, number=100000)

# === Mesurer une fonction ===
def my_function():
    return sum(range(1000))

# Méthode 1: Passer fonction directement
result = timeit.timeit(my_function, number=10000)

# Méthode 2: Avec lambda
result = timeit.timeit(lambda: my_function(), number=10000)

# Méthode 3: String avec import
result = timeit.timeit(
    'my_function()',
    setup='from __main__ import my_function',
    number=10000
)

# === repeat() - Plusieurs runs ===
results = timeit.repeat(
    'sum(range(1000))',
    repeat=5,      # 5 runs
    number=10000   # 10000 exécutions par run
)
print(f"Min: {min(results):.6f}s")
print(f"Avg: {sum(results)/len(results):.6f}s")

# === Ligne de commande ===
# python -m timeit '"-".join(str(n) for n in range(100))'
# python -m timeit -n 10000 -r 5 'sum(range(1000))'
# python -m timeit -s 'from math import sqrt' 'sqrt(144)'

# === Comparer plusieurs approches ===
def compare_approaches():
    """Compare différentes méthodes"""
    
    # Méthode 1: List comprehension
    time1 = timeit.timeit(
        '[x**2 for x in range(1000)]',
        number=10000
    )
    
    # Méthode 2: map
    time2 = timeit.timeit(
        'list(map(lambda x: x**2, range(1000)))',
        number=10000
    )
    
    # Méthode 3: Loop
    time3 = timeit.timeit(
        '''
result = []
for x in range(1000):
    result.append(x**2)
        ''',
        number=10000
    )
    
    print(f"List comp: {time1:.6f}s")
    print(f"Map:       {time2:.6f}s")
    print(f"Loop:      {time3:.6f}s")

# === Benchmark avec warmup ===
def benchmark_with_warmup(func, warmup=3, runs=100):
    """Benchmark avec période de chauffe"""
    # Warmup
    for _ in range(warmup):
        func()
    
    # Mesure
    times = []
    for _ in range(runs):
        start = time.perf_counter()
        func()
        times.append(time.perf_counter() - start)
    
    return {
        'min': min(times),
        'max': max(times),
        'mean': sum(times) / len(times),
        'median': sorted(times)[len(times)//2]
    }


[OK] CPROFILE - PROFILING DÉTAILLÉ

# === Usage basique ===
import cProfile

# Profiler une fonction
cProfile.run('my_function()')

# Profiler un script
# python -m cProfile script.py
# python -m cProfile -s cumulative script.py  # Trié par temps cumulé

# === Sauvegarder et analyser ===
import pstats

# Sauvegarder résultats
cProfile.run('my_function()', 'profile_stats')

# Analyser
p = pstats.Stats('profile_stats')

# Trier par temps cumulé
p.sort_stats('cumulative')
p.print_stats(10)  # Top 10

# Autres tris
p.sort_stats('time')        # Temps propre
p.sort_stats('calls')       # Nombre d'appels
p.sort_stats('name')        # Nom fonction

# Filtrer
p.print_stats('mymodule')   # Seulement mymodule
p.print_stats(0.1)          # Top 10%

# === Profiler contexte spécifique ===
import cProfile
import pstats
from io import StringIO

def profile_code(func):
    """Profile fonction et retourne stats"""
    profiler = cProfile.Profile()
    profiler.enable()
    
    result = func()
    
    profiler.disable()
    
    s = StringIO()
    ps = pstats.Stats(profiler, stream=s)
    ps.sort_stats('cumulative')
    ps.print_stats()
    
    print(s.getvalue())
    return result

# === Décorateur de profiling ===
def profile(func):
    """Décorateur pour profiler fonction"""
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        profiler = cProfile.Profile()
        profiler.enable()
        result = func(*args, **kwargs)
        profiler.disable()
        
        stats = pstats.Stats(profiler)
        stats.sort_stats('cumulative')
        stats.print_stats(20)
        
        return result
    return wrapper

@profile
def expensive_function():
    # Code coûteux
    pass

# === Analyser résultats ===
"""
Colonnes importantes:
- ncalls: Nombre d'appels
- tottime: Temps total dans fonction (sans sous-appels)
- cumtime: Temps cumulé (avec sous-appels)
- percall: Temps par appel
- filename:lineno: Localisation

Exemple output:
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     1000    0.008    0.000    0.050    0.000 module.py:10(func)
"""

# === SnakeViz - Visualisation graphique ===
# pip install snakeviz
# python -m cProfile -o profile.stats script.py
# snakeviz profile.stats


[OK] LINE_PROFILER - PROFILING LIGNE PAR LIGNE

# pip install line_profiler

# === Méthode 1: Décorateur ===
# Fichier: script.py

@profile  # Le décorateur @profile est magique, pas besoin d'import
def slow_function():
    total = 0
    for i in range(1000):
        total += i ** 2
    
    results = []
    for i in range(1000):
        results.append(i)
    
    return total, results

if __name__ == '__main__':
    slow_function()

# Exécuter:
# kernprof -l -v script.py
# -l: line-by-line
# -v: verbose (affiche résultats)

# === Méthode 2: Programmatique ===
from line_profiler import LineProfiler

def slow_function():
    total = 0
    for i in range(1000):
        total += i ** 2
    return total

profiler = LineProfiler()
profiler.add_function(slow_function)
profiler.enable()

slow_function()

profiler.disable()
profiler.print_stats()

# === Profiler plusieurs fonctions ===
profiler = LineProfiler()
profiler.add_function(func1)
profiler.add_function(func2)
profiler.add_function(func3)
# ...

# === Interpréter résultats ===
"""
Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
     1                                           def slow_function():
     2         1          2.0      2.0      0.1      total = 0
     3      1001        300.0      0.3     15.0      for i in range(1000):
     4      1000       1700.0      1.7     85.0          total += i ** 2
     5         1          0.0      0.0      0.0      return total

Colonne importante: % Time - où le temps est passé
"""


[OK] MEMORY_PROFILER - PROFILING MÉMOIRE

# pip install memory_profiler

# === Décorateur ===
from memory_profiler import profile

@profile
def memory_intensive():
    a = [1] * (10 ** 6)           # ~8 MB
    b = [2] * (2 * 10 ** 7)       # ~160 MB
    del b                          # Libère
    return a

if __name__ == '__main__':
    memory_intensive()

# Exécuter:
# python -m memory_profiler script.py

# === mprof - Graphique temporel ===
# mprof run script.py
# mprof plot

# === memory_usage() - Mesurer usage ===
from memory_profiler import memory_usage

def my_function():
    a = [1] * (10 ** 7)
    return sum(a)

# Mesurer usage maximum
mem_usage = memory_usage((my_function, (), {}))
print(f"Memory usage: {max(mem_usage):.2f} MB")

# Avec interval
mem_usage = memory_usage(
    (my_function, (), {}),
    interval=0.1,      # Mesure chaque 0.1s
    timeout=10         # Max 10s
)

# === Comparer usage mémoire ===
import tracemalloc

def compare_memory():
    # Démarrer traçage
    tracemalloc.start()
    
    # Code à profiler
    data = [x ** 2 for x in range(10000)]
    
    # Snapshot
    snapshot = tracemalloc.take_snapshot()
    top_stats = snapshot.statistics('lineno')
    
    # Top 10 allocations
    for stat in top_stats[:10]:
        print(stat)
    
    tracemalloc.stop()

# === Tracker mémoire personnalisé ===
import sys

def get_size(obj):
    """Taille objet en bytes"""
    return sys.getsizeof(obj)

# Comparaison structures
list_size = sys.getsizeof([x for x in range(1000)])
gen_size = sys.getsizeof(x for x in range(1000))
print(f"List: {list_size} bytes")
print(f"Generator: {gen_size} bytes")


[OK] OPTIMISATIONS GÉNÉRALES

# === 1. List Comprehension vs Loop ===

# [X] Lent
result = []
for i in range(1000):
    result.append(i ** 2)

# [OK] Rapide (2-3x plus rapide)
result = [i ** 2 for i in range(1000)]

# [OK] Generator pour économie mémoire
result = (i ** 2 for i in range(1000000))

# Benchmark
import timeit

time_loop = timeit.timeit('''
result = []
for i in range(1000):
    result.append(i ** 2)
''', number=10000)

time_comp = timeit.timeit(
    'result = [i ** 2 for i in range(1000)]',
    number=10000
)

print(f"Loop: {time_loop:.4f}s")
print(f"Comp: {time_comp:.4f}s")
print(f"Speedup: {time_loop/time_comp:.2f}x")


# === 2. String Concatenation ===

# [X] Très lent (O(n²))
strings = ['a'] * 10000
result = ""
for s in strings:
    result += s

# [OK] Rapide (O(n))
result = "".join(strings)

# [OK] Pour petites strings (<10)
result = s1 + s2 + s3  # OK

# StringBuilder pattern
parts = []
for item in items:
    parts.append(str(item))
result = "".join(parts)

# f-strings vs %s vs format()
# f-strings généralement plus rapides
name = "Alice"
age = 30

# Rapide
s = f"{name} is {age}"

# Moyen
s = "%s is %d" % (name, age)

# Lent
s = "{} is {}".format(name, age)


# === 3. Membership Testing ===

# [X] Lent (O(n))
items = list(range(10000))
if 5000 in items:  # Parcourt liste
    pass

# [OK] Rapide (O(1))
items = set(range(10000))
if 5000 in items:  # Lookup constant
    pass

# Comparaison
import timeit

items_list = list(range(10000))
items_set = set(range(10000))

time_list = timeit.timeit(
    '5000 in items',
    setup='items = list(range(10000))',
    number=10000
)

time_set = timeit.timeit(
    '5000 in items',
    setup='items = set(range(10000))',
    number=10000
)

print(f"List: {time_list:.6f}s")
print(f"Set:  {time_set:.6f}s")
print(f"Speedup: {time_list/time_set:.0f}x")


# === 4. Éviter Lookups Répétés ===

# [X] Lent - lookup répété
for i in range(1000):
    result = math.sqrt(i)

# [OK] Mieux - import local
from math import sqrt
for i in range(1000):
    result = sqrt(i)

# [OK] Encore mieux - variable locale
from math import sqrt
sqrt_func = sqrt
for i in range(1000):
    result = sqrt_func(i)

# Attribute lookup
# [X] Lent
for item in items:
    item.attribute.method()

# [OK] Rapide
for item in items:
    method = item.attribute.method
    method()

# List append optimization
# [X] Lent
result = []
append = result.append
for i in range(10000):
    append(i)  # Évite lookup


# === 5. Built-in Functions ===

# Built-ins sont en C, donc très rapides

# [X] Lent
total = 0
for num in numbers:
    total += num

# [OK] Rapide
total = sum(numbers)

# [X] Lent
result = []
for item in items:
    if condition(item):
        result.append(item)

# [OK] Rapide
result = list(filter(condition, items))
# ou
result = [item for item in items if condition(item)]

# map() vs list comprehension
# map légèrement plus rapide mais moins lisible
squared = map(lambda x: x**2, numbers)  # Lazy
squared = [x**2 for x in numbers]       # Eager, plus pythonic


# === 6. Local Variables ===

# Variables locales plus rapides que globales

GLOBAL_VAR = 100

def with_global():
    total = 0
    for i in range(1000):
        total += GLOBAL_VAR  # Lent

def with_local():
    local_var = GLOBAL_VAR
    total = 0
    for i in range(1000):
        total += local_var  # Rapide


# === 7. Loop Optimization ===

# Éviter calculs dans condition de loop
# [X] Lent
for i in range(len(items)):  # len() appelé chaque fois
    process(items[i])

# [OK] Rapide
length = len(items)
for i in range(length):
    process(items[i])

# [OK] Encore mieux
for item in items:
    process(item)

# Inverser loops si possible
# [X] Lent - test dans loop
for i in range(1000):
    if condition:
        process_a(i)
    else:
        process_b(i)

# [OK] Rapide - test hors loop
if condition:
    for i in range(1000):
        process_a(i)
else:
    for i in range(1000):
        process_b(i)


# === 8. Dict Operations ===

# get() vs KeyError
# get() généralement plus rapide
value = my_dict.get(key, default)

# Mais try/except plus rapide si clé existe souvent (EAFP)
try:
    value = my_dict[key]
except KeyError:
    value = default

# setdefault vs if-else
# [X] Lent
if key not in my_dict:
    my_dict[key] = []
my_dict[key].append(value)

# [OK] Rapide
my_dict.setdefault(key, []).append(value)

# [OK] Ou defaultdict
from collections import defaultdict
my_dict = defaultdict(list)
my_dict[key].append(value)

# Dict comprehension
# [OK] Rapide
result = {k: v for k, v in items}

# Multiple dict access
# [X] Lent
for key in keys:
    value1 = dict1[key]
    value2 = dict2[key]
    process(value1, value2)

# [OK] Rapide - cache get method
get1 = dict1.get
get2 = dict2.get
for key in keys:
    value1 = get1(key)
    value2 = get2(key)
    process(value1, value2)


[OK] STRUCTURES DE DONNÉES OPTIMISÉES

# === collections ===

from collections import deque, Counter, defaultdict, namedtuple

# deque - Queue optimisée
# [X] Lent - list pour queue
queue = []
queue.append(1)      # O(1)
queue.pop(0)         # O(n) !

# [OK] Rapide - deque
queue = deque()
queue.append(1)      # O(1)
queue.popleft()      # O(1)

# Comparaison
import timeit

time_list = timeit.timeit('''
q = []
for i in range(1000):
    q.append(i)
for i in range(1000):
    q.pop(0)
''', number=1000)

time_deque = timeit.timeit('''
from collections import deque
q = deque()
for i in range(1000):
    q.append(i)
for i in range(1000):
    q.popleft()
''', number=1000)

print(f"Speedup: {time_list/time_deque:.1f}x")


# Counter - Comptage optimisé
# [X] Lent
counts = {}
for item in items:
    counts[item] = counts.get(item, 0) + 1

# [OK] Rapide
counts = Counter(items)

# Opérations Counter
c = Counter(['a', 'b', 'a', 'c', 'b', 'a'])
print(c.most_common(2))  # [('a', 3), ('b', 2)]
print(c['a'])            # 3


# defaultdict - Évite KeyError
# [X] Lent
groups = {}
for key, value in items:
    if key not in groups:
        groups[key] = []
    groups[key].append(value)

# [OK] Rapide
groups = defaultdict(list)
for key, value in items:
    groups[key].append(value)


# namedtuple - Alternative légère aux classes
# [X] Plus lourd
class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y

# [OK] Plus léger
Point = namedtuple('Point', ['x', 'y'])
p = Point(1, 2)
print(p.x, p.y)


# === array - Arrays typés ===
from array import array

# Plus efficace que list pour nombres uniformes
# [X] List - 8 bytes par élément + overhead
numbers_list = [1, 2, 3, 4, 5] * 1000

# [OK] Array - type size par élément
numbers_array = array('i', [1, 2, 3, 4, 5] * 1000)  # 'i' = int (4 bytes)

# Types disponibles
# 'b': signed char (1 byte)
# 'i': signed int (4 bytes)
# 'f': float (4 bytes)
# 'd': double (8 bytes)

import sys
print(f"List size: {sys.getsizeof(numbers_list)} bytes")
print(f"Array size: {sys.getsizeof(numbers_array)} bytes")


# === bisect - Binary search ===
import bisect

# Maintenir liste triée
sorted_list = []
for value in values:
    bisect.insort(sorted_list, value)  # O(n) mais liste reste triée

# Recherche rapide
index = bisect.bisect_left(sorted_list, value)  # O(log n)


# === heapq - Priority queue ===
import heapq

# Min heap
heap = []
heapq.heappush(heap, 3)
heapq.heappush(heap, 1)
heapq.heappush(heap, 2)

smallest = heapq.heappop(heap)  # 1

# n plus petits/grands éléments
numbers = [5, 2, 8, 1, 9, 3]
smallest_3 = heapq.nsmallest(3, numbers)  # [1, 2, 3]
largest_3 = heapq.nlargest(3, numbers)    # [9, 8, 5]


[OK] SLOTS POUR CLASSES

# __slots__ réduit utilisation mémoire et accélère attributs

# [X] Sans __slots__ - utilise __dict__
class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y

# [OK] Avec __slots__ - pas de __dict__
class Point:
    __slots__ = ['x', 'y']
    
    def __init__(self, x, y):
        self.x = x
        self.y = y

# Comparaison mémoire
import sys

class WithoutSlots:
    def __init__(self, x, y):
        self.x = x
        self.y = y

class WithSlots:
    __slots__ = ['x', 'y']
    def __init__(self, x, y):
        self.x = x
        self.y = y

p1 = WithoutSlots(1, 2)
p2 = WithSlots(1, 2)

print(f"Sans slots: {sys.getsizeof(p1) + sys.getsizeof(p1.__dict__)} bytes")
print(f"Avec slots: {sys.getsizeof(p2)} bytes")

# Économie: ~50% mémoire pour millions d'instances

# Limitations __slots__:
# - Ne peut pas ajouter attributs dynamiquement
# - Héritage plus complexe
# - Pas de __dict__ (certains outils en ont besoin)


[OK] FUNCTOOLS - OPTIMISATIONS FONCTIONNELLES

# === lru_cache - Memoization ===
from functools import lru_cache

# Sans cache - très lent
def fibonacci(n):
    if n < 2:
        return n
    return fibonacci(n-1) + fibonacci(n-2)

# Avec cache - très rapide
@lru_cache(maxsize=128)
def fibonacci_cached(n):
    if n < 2:
        return n
    return fibonacci_cached(n-1) + fibonacci_cached(n-2)

# Benchmark
import timeit
print(f"Sans cache: {timeit.timeit(lambda: fibonacci(30), number=1):.4f}s")
print(f"Avec cache: {timeit.timeit(lambda: fibonacci_cached(30), number=1):.6f}s")

# maxsize=None pour cache illimité
@lru_cache(maxsize=None)
def expensive_function(x):
    # Calcul coûteux
    return x ** 10

# Statistiques cache
print(fibonacci_cached.cache_info())
# CacheInfo(hits=..., misses=..., maxsize=128, currsize=...)

# Vider cache
fibonacci_cached.cache_clear()


# === cache (Python 3.9+) ===
from functools import cache

@cache  # Équivalent à lru_cache(maxsize=None)
def factorial(n):
    return n * factorial(n-1) if n else 1


# === cached_property ===
from functools import cached_property

class DataProcessor:
    def __init__(self, data):
        self.data = data
    
    @cached_property
    def expensive_computation(self):
        """Calculé une seule fois puis caché"""
        print("Computing...")
        return sum(x ** 2 for x in self.data)

processor = DataProcessor(range(1000))
print(processor.expensive_computation)  # Computing... 332833500
print(processor.expensive_computation)  # 332833500 (pas de Computing)


# === partial - Pré-application arguments ===
from functools import partial

def power(base, exponent):
    return base ** exponent

# Créer fonction spécialisée
square = partial(power, exponent=2)
cube = partial(power, exponent=3)

print(square(5))  # 25
print(cube(5))    # 125


# === reduce - Éviter loops ===
from functools import reduce

# [X] Lent
total = 0
for num in numbers:
    total += num

# [OK] Rapide
total = sum(numbers)  # Built-in préféré

# reduce pour opérations custom
product = reduce(lambda x, y: x * y, numbers, 1)


[OK] ITERTOOLS - ITÉRATION EFFICACE

from itertools import *

# === islice - Slice sans matérialisation ===
# [X] Crée liste complète
big_range = list(range(1000000))
subset = big_range[100:200]

# [OK] Pas de matérialisation
subset = list(islice(range(1000000), 100, 200))


# === chain - Chaîner itérables ===
# [X] Crée liste intermédiaire
combined = list1 + list2 + list3

# [OK] Pas de copie
combined = chain(list1, list2, list3)


# === groupby - Grouper éléments ===
from itertools import groupby

data = [
    {'type': 'A', 'value': 1},
    {'type': 'A', 'value': 2},
    {'type': 'B', 'value': 3},
    {'type': 'B', 'value': 4},
]

# Grouper (nécessite données triées par clé)
for key, group in groupby(data, key=lambda x: x['type']):
    print(key, list(group))


# === compress - Filtrage avec sélecteur ===
data = ['A', 'B', 'C', 'D', 'E']
selectors = [1, 0, 1, 0, 1]
result = list(compress(data, selectors))  # ['A', 'C', 'E']


# === accumulate - Accumulation ===
numbers = [1, 2, 3, 4, 5]
cumsum = list(accumulate(numbers))  # [1, 3, 6, 10, 15]

# Avec fonction custom
import operator
cumproduct = list(accumulate(numbers, operator.mul))  # [1, 2, 6, 24, 120]


# === product - Produit cartésien ===
# [X] Nested loops
result = []
for a in [1, 2]:
    for b in ['x', 'y']:
        result.append((a, b))

# [OK] product
result = list(product([1, 2], ['x', 'y']))
# [(1,'x'), (1,'y'), (2,'x'), (2,'y')]


# === combinations & permutations ===
from itertools import combinations, permutations

items = [1, 2, 3]
print(list(combinations(items, 2)))    # [(1,2), (1,3), (2,3)]
print(list(permutations(items, 2)))    # [(1,2), (1,3), (2,1), (2,3), (3,1), (3,2)]


[OK] NUMPY - CALCUL VECTORISÉ

# pip install numpy

import numpy as np

# === Opérations vectorisées ===

# [X] Python pur - très lent
def python_sum_squares(n):
    result = []
    for i in range(n):
        result.append(i ** 2)
    return sum(result)

# [OK] NumPy - très rapide (10-100x)
def numpy_sum_squares(n):
    arr = np.arange(n)
    return np.sum(arr ** 2)

# Benchmark
import timeit
n = 1000000

time_python = timeit.timeit(lambda: python_sum_squares(n), number=10)
time_numpy = timeit.timeit(lambda: numpy_sum_squares(n), number=10)

print(f"Python: {time_python:.4f}s")
print(f"NumPy:  {time_numpy:.4f}s")
print(f"Speedup: {time_python/time_numpy:.1f}x")


# === Opérations élémentaires ===
arr = np.array([1, 2, 3, 4, 5])

# Arithmétique vectorisée
result = arr * 2              # [2, 4, 6, 8, 10]
result = arr ** 2             # [1, 4, 9, 16, 25]
result = np.sqrt(arr)         # [1.0, 1.41, 1.73, 2.0, 2.24]

# Opérations matricielles
matrix1 = np.random.rand(1000, 1000)
matrix2 = np.random.rand(1000, 1000)

# Multiplication matricielle (très rapide avec BLAS/LAPACK)
result = np.dot(matrix1, matrix2)
# ou
result = matrix1 @ matrix2


# === Broadcasting ===
# Opérations sur arrays de tailles différentes

arr = np.array([[1, 2, 3],
                [4, 5, 6]])

# Ajouter vecteur à chaque ligne
vec = np.array([10, 20, 30])
result = arr + vec  # Broadcasting automatique
# [[11, 22, 33],
#  [14, 25, 36]]


# === Universal functions (ufuncs) ===
# Fonctions optimisées en C

arr = np.arange(1000000)

# [X] Python
result = [x ** 2 for x in arr]

# [OK] NumPy ufunc
result = np.square(arr)

# Autres ufuncs utiles
np.sqrt(arr)
np.exp(arr)
np.log(arr)
np.sin(arr)
np.abs(arr)


# === Réduction d'axes ===
matrix = np.random.rand(1000, 1000)

# Somme totale
total = np.sum(matrix)

# Somme par colonne
col_sums = np.sum(matrix, axis=0)

# Somme par ligne
row_sums = np.sum(matrix, axis=1)

# Autres réductions
np.mean(matrix, axis=0)
np.max(matrix, axis=1)
np.min(matrix)
np.std(matrix)


# === Indexation avancée ===
arr = np.arange(10)

# Boolean indexing
mask = arr > 5
result = arr[mask]  # [6, 7, 8, 9]

# Fancy indexing
indices = [1, 3, 5]
result = arr[indices]  # [1, 3, 5]


# === Éviter copies ===
# [X] Crée copie
arr_copy = arr.copy()

# [OK] Vue (pas de copie)
arr_view = arr[::2]  # Vue sur éléments pairs

# Attention: modifier vue modifie original
arr_view[0] = 999
print(arr[0])  # 999


# === Optimisations mémoire ===
# Spécifier dtype pour économiser mémoire

# [X] float64 par défaut (8 bytes)
arr = np.random.rand(1000000)

# [OK] float32 suffit souvent (4 bytes)
arr = np.random.rand(1000000).astype(np.float32)

# Types disponibles
# int8, int16, int32, int64
# uint8, uint16, uint32, uint64
# float16, float32, float64
# bool (1 bit)


[OK] NUMBA - JIT COMPILATION

# pip install numba

from numba import jit, njit, prange

# === Compilation JIT basique ===

# Python pur - lent
def slow_function(n):
    total = 0
    for i in range(n):
        total += i ** 2
    return total

# Avec Numba - rapide
@jit
def fast_function(n):
    total = 0
    for i in range(n):
        total += i ** 2
    return total

# Benchmark
import timeit
n = 10000000

time_python = timeit.timeit(lambda: slow_function(n), number=10)
time_numba = timeit.timeit(lambda: fast_function(n), number=10)

print(f"Python: {time_python:.4f}s")
print(f"Numba:  {time_numba:.4f}s")
print(f"Speedup: {time_python/time_numba:.0f}x")


# === njit - No Python mode (plus rapide) ===
@njit
def very_fast_function(n):
    total = 0
    for i in range(n):
        total += i ** 2
    return total


# === Parallel loops ===
@njit(parallel=True)
def parallel_sum(arr):
    total = 0
    # prange = parallel range
    for i in prange(len(arr)):
        total += arr[i] ** 2
    return total

# Utilise tous les cores CPU


# === Cache compilation ===
@njit(cache=True)
def cached_function(x):
    # Compilation cachée sur disque
    # Pas de recompilation au prochain run
    return x ** 2


# === Signatures de type ===
from numba import int64, float64

@jit(int64(int64, int64))
def add(x, y):
    return x + y

# Ou plusieurs signatures
@jit([int64(int64, int64), float64(float64, float64)])
def add_multi(x, y):
    return x + y


# === Quand utiliser Numba ===
# [OK] Boucles intensives
# [OK] Calculs numériques
# [OK] Pas de structures Python complexes
# [X] Manipulation strings
# [X] Classes complexes
# [X] I/O operations


[OK] CYTHON - COMPILATION C

# pip install cython

# === Fichier .pyx ===
# mymodule.pyx

def python_sum(n):
    """Python pur"""
    total = 0
    for i in range(n):
        total += i
    return total

def cython_sum(int n):
    """Avec types C"""
    cdef int i
    cdef long total = 0
    for i in range(n):
        total += i
    return total

# setup.py
from setuptools import setup
from Cython.Build import cythonize

setup(
    ext_modules=cythonize("mymodule.pyx")
)

# Compiler:
# python setup.py build_ext --inplace


# === Types statiques ===
# cdef pour variables locales rapides
def fast_function():
    cdef int i, n = 1000
    cdef double result = 0.0
    
    for i in range(n):
        result += i * 0.5
    
    return result


# === Numpy integration ===
import numpy as np
cimport numpy as cnp

def cython_numpy_sum(cnp.ndarray[cnp.float64_t, ndim=1] arr):
    cdef int i, n = arr.shape[0]
    cdef double total = 0.0
    
    for i in range(n):
        total += arr[i]
    
    return total


# === Quand utiliser Cython ===
# [OK] Performance critique
# [OK] Extension C libraries
# [OK] NumPy operations custom
# [X] Complexité ajoutée
# [X] Build process


[OK] MULTIPROCESSING - PARALLÉLISME CPU

from multiprocessing import Pool, Process, Queue, Manager
import multiprocessing as mp

# === Pool basique ===
def worker(x):
    """Fonction worker"""
    return x ** 2

if __name__ == '__main__':
    # Pool de 4 workers
    with Pool(4) as pool:
        results = pool.map(worker, range(10))
        print(results)  # [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]


# === map vs imap ===
# map: attend tous les résultats
results = pool.map(worker, data)

# imap: retourne itérateur lazy
for result in pool.imap(worker, data):
    print(result)  # Traite au fur et à mesure

# imap_unordered: plus rapide mais désordre
for result in pool.imap_unordered(worker, data):
    print(result)


# === starmap - Multiple arguments ===
def worker_multi(x, y):
    return x + y

data = [(1, 2), (3, 4), (5, 6)]
results = pool.starmap(worker_multi, data)  # [3, 7, 11]


# === apply_async - Asynchrone ===
def callback(result):
    print(f"Got result: {result}")

result = pool.apply_async(worker, (5,), callback=callback)
result.wait()  # Attendre fin


# === Process manuel ===
def process_function(name):
    print(f"Process {name} running")

if __name__ == '__main__':
    processes = []
    for i in range(4):
        p = Process(target=process_function, args=(f"Worker-{i}",))
        p.start()
        processes.append(p)
    
    for p in processes:
        p.join()


# === Queue pour communication ===
def producer(queue):
    for i in range(10):
        queue.put(i)
    queue.put(None)  # Sentinel

def consumer(queue):
    while True:
        item = queue.get()
        if item is None:
            break
        print(f"Processing {item}")

if __name__ == '__main__':
    q = Queue()
    p1 = Process(target=producer, args=(q,))
    p2 = Process(target=consumer, args=(q,))
    
    p1.start()
    p2.start()
    
    p1.join()
    p2.join()


# === Manager - Shared state ===
def worker_shared(shared_dict, key, value):
    shared_dict[key] = value

if __name__ == '__main__':
    manager = Manager()
    shared = manager.dict()
    
    processes = []
    for i in range(4):
        p = Process(target=worker_shared, args=(shared, i, i**2))
        p.start()
        processes.append(p)
    
    for p in processes:
        p.join()
    
    print(dict(shared))


# === Optimal worker count ===
# CPU-bound
num_workers = mp.cpu_count()  # Tous les cores

# I/O-bound
num_workers = mp.cpu_count() * 2  # Plus de workers OK


# === Benchmark multiprocessing ===
import time

def expensive_task(n):
    return sum(i ** 2 for i in range(n))

data = [1000000] * 8

# Séquentiel
start = time.time()
results = [expensive_task(n) for n in data]
seq_time = time.time() - start

# Parallel
start = time.time()
with Pool(4) as pool:
    results = pool.map(expensive_task, data)
parallel_time = time.time() - start

print(f"Sequential: {seq_time:.2f}s")
print(f"Parallel:   {parallel_time:.2f}s")
print(f"Speedup:    {seq_time/parallel_time:.1f}x")


[OK] CONCURRENT.FUTURES - PARALLÉLISME SIMPLIFIÉ

from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
from concurrent.futures import as_completed, wait

# === ThreadPoolExecutor (I/O-bound) ===
import requests

def fetch_url(url):
    response = requests.get(url)
    return len(response.content)

urls = ['http://example.com'] * 10

# Avec threads
with ThreadPoolExecutor(max_workers=5) as executor:
    results = executor.map(fetch_url, urls)
    print(list(results))


# === ProcessPoolExecutor (CPU-bound) ===
def cpu_intensive(n):
    return sum(i ** 2 for i in range(n))

data = [1000000] * 8

# Avec processes
with ProcessPoolExecutor(max_workers=4) as executor:
    results = executor.map(cpu_intensive, data)
    print(list(results))


# === submit() - Plus de contrôle ===
with ThreadPoolExecutor(max_workers=5) as executor:
    # Soumettre tâches
    futures = [executor.submit(fetch_url, url) for url in urls]
    
    # Attendre et récupérer résultats
    for future in as_completed(futures):
        try:
            result = future.result()
            print(f"Got result: {result}")
        except Exception as e:
            print(f"Error: {e}")


# === Timeout ===
with ThreadPoolExecutor() as executor:
    future = executor.submit(slow_function)
    try:
        result = future.result(timeout=5)  # Max 5 secondes
    except TimeoutError:
        print("Timeout!")


# === Callbacks ===
def done_callback(future):
    print(f"Task completed: {future.result()}")

with ThreadPoolExecutor() as executor:
    future = executor.submit(fetch_url, 'http://example.com')
    future.add_done_callback(done_callback)


# === Comparaison threads vs processes ===
"""
ThreadPoolExecutor:
[OK] I/O-bound (network, files)
[OK] Overhead minimal
[OK] Shared memory
[X] GIL limite CPU-bound

ProcessPoolExecutor:
[OK] CPU-bound
[OK] Bypass GIL
[X] Overhead important (IPC)
[X] Serialization nécessaire
"""


[OK] ASYNCIO - PROGRAMMATION ASYNCHRONE

import asyncio
import aiohttp
import aiofiles

# === Fonction async basique ===
async def async_function():
    print("Start")
    await asyncio.sleep(1)  # I/O async
    print("End")
    return "Result"

# Exécuter
asyncio.run(async_function())


# === Requêtes HTTP async ===
# pip install aiohttp

async def fetch_url(session, url):
    async with session.get(url) as response:
        return await response.text()

async def fetch_multiple(urls):
    async with aiohttp.ClientSession() as session:
        tasks = [fetch_url(session, url) for url in urls]
        results = await asyncio.gather(*tasks)
        return results

# Utilisation
urls = ['http://example.com'] * 100
results = asyncio.run(fetch_multiple(urls))


# === Lecture fichier async ===
# pip install aiofiles

async def read_file(filename):
    async with aiofiles.open(filename, 'r') as f:
        content = await f.read()
        return content

async def read_multiple_files(filenames):
    tasks = [read_file(f) for f in filenames]
    return await asyncio.gather(*tasks)


# === Semaphore - Limiter concurrence ===
async def limited_fetch(session, url, semaphore):
    async with semaphore:
        # Max N requêtes simultanées
        return await fetch_url(session, url)

async def fetch_with_limit(urls, max_concurrent=10):
    semaphore = asyncio.Semaphore(max_concurrent)
    async with aiohttp.ClientSession() as session:
        tasks = [limited_fetch(session, url, semaphore) for url in urls]
        return await asyncio.gather(*tasks)


# === Queue async ===
async def producer(queue):
    for i in range(10):
        await queue.put(i)
        await asyncio.sleep(0.1)

async def consumer(queue):
    while True:
        item = await queue.get()
        print(f"Processing {item}")
        queue.task_done()

async def main():
    queue = asyncio.Queue()
    
    # Démarrer producer et consumers
    prod = asyncio.create_task(producer(queue))
    cons = [asyncio.create_task(consumer(queue)) for _ in range(3)]
    
    await prod
    await queue.join()
    
    # Annuler consumers
    for c in cons:
        c.cancel()

asyncio.run(main())


# === Timeout async ===
async def slow_operation():
    await asyncio.sleep(10)
    return "Done"

async def with_timeout():
    try:
        result = await asyncio.wait_for(slow_operation(), timeout=5)
    except asyncio.TimeoutError:
        print("Timeout!")


# === Benchmark sync vs async ===
import time
import requests

# Synchrone
def sync_fetch(urls):
    results = []
    for url in urls:
        response = requests.get(url)
        results.append(len(response.content))
    return results

# Asynchrone
async def async_fetch(urls):
    async with aiohttp.ClientSession() as session:
        tasks = [fetch_url(session, url) for url in urls]
        results = await asyncio.gather(*tasks)
        return [len(r) for r in results]

urls = ['http://httpbin.org/delay/1'] * 10

# Test sync
start = time.time()
sync_fetch(urls)
sync_time = time.time() - start

# Test async
start = time.time()
asyncio.run(async_fetch(urls))
async_time = time.time() - start

print(f"Sync:  {sync_time:.2f}s")
print(f"Async: {async_time:.2f}s")
print(f"Speedup: {sync_time/async_time:.1f}x")


[OK] OPTIMISATIONS AVANCÉES

# === 1. Lazy Evaluation ===

# [X] Eager - calcule tout
def eager_filter_map(data):
    filtered = [x for x in data if x > 0]
    mapped = [x ** 2 for x in filtered]
    return mapped

# [OK] Lazy - génère à la demande
def lazy_filter_map(data):
    filtered = (x for x in data if x > 0)
    mapped = (x ** 2 for x in filtered)
    return mapped


# === 2. Avoid Repeated Computation ===

# [X] Calcule plusieurs fois
class BadClass:
    def __init__(self, data):
        self.data = data
    
    def process(self):
        # Calcule sum à chaque appel!
        if sum(self.data) > 100:
            return "High"
        return "Low"

# [OK] Cache résultat
class GoodClass:
    def __init__(self, data):
        self.data = data
        self._sum_cache = None
    
    @property
    def data_sum(self):
        if self._sum_cache is None:
            self._sum_cache = sum(self.data)
        return self._sum_cache
    
    def process(self):
        if self.data_sum > 100:
            return "High"
        return "Low"


# === 3. String Interning ===

# Python interne automatiquement petites strings
# Mais pas les grandes ou dynamiques

# Forcer interning pour comparaisons fréquentes
import sys

s1 = sys.intern("very long string that will be compared many times")
s2 = sys.intern("very long string that will be compared many times")

# Maintenant s1 is s2 (identity check, ultra rapide)
assert s1 is s2


# === 4. Préallocation ===

# [X] List grows dynamically
result = []
for i in range(10000):
    result.append(i ** 2)

# [OK] Préallouer (légèrement plus rapide)
result = [0] * 10000
for i in range(10000):
    result[i] = i ** 2

# [OK] Encore mieux: list comprehension
result = [i ** 2 for i in range(10000)]


# === 5. ByteArray pour modifications ===

# [X] Strings immutables - lent pour modifications
s = "hello"
s += " world"  # Crée nouveau string

# [OK] ByteArray mutable
ba = bytearray(b"hello")
ba.extend(b" world")  # Modifie in-place


# === 6. Pickle Protocol ===

import pickle

# Protocole 4 (Python 3.4+) plus rapide
data = list(range(1000000))

# Lent (protocol 0)
pickled = pickle.dumps(data, protocol=0)

# Rapide (protocol 4)
pickled = pickle.dumps(data, protocol=4)

# Le plus rapide (protocol le plus récent)
pickled = pickle.dumps(data, protocol=pickle.HIGHEST_PROTOCOL)


# === 7. Avoid Global Lookups ===

import math

# [X] Global lookup dans loop
def slow():
    result = 0
    for i in range(1000):
        result += math.sqrt(i)
    return result

# [OK] Local reference
def fast():
    sqrt = math.sqrt  # Local
    result = 0
    for i in range(1000):
        result += sqrt(i)
    return result


# === 8. Use __slots__ for Data Classes ===

from dataclasses import dataclass

# [X] Sans slots
@dataclass
class Point:
    x: float
    y: float

# [OK] Avec slots
@dataclass
class Point:
    __slots__ = ['x', 'y']
    x: float
    y: float


[OK] PROFILING TOOLS AVANCÉS

# === py-spy - Sampling Profiler ===
# pip install py-spy

# Profiler process en cours
# py-spy top --pid 12345

# Générer flamegraph
# py-spy record -o profile.svg -- python script.py

# Avantages:
# - Pas besoin modifier code
# - Overhead minimal
# - Profiling production


# === scalene - CPU + GPU + Memory ===
# pip install scalene

# scalene script.py

# Fonctionnalités:
# - Profiling CPU par ligne
# - Profiling mémoire
# - GPU usage (si applicable)
# - Suggestions optimisation


# === viztracer - Tracing visuel ===
# pip install viztracer

# viztracer script.py
# vizviewer result.json

# Visualise:
# - Timeline exécution
# - Call stack
# - Durée fonctions


# === memray - Memory profiler avancé ===
# pip install memray

# python -m memray run script.py
# python -m memray flamegraph output.bin

# Fonctionnalités:
# - Memory leaks
# - Allocation patterns
# - Peak memory


# === austin - Frame stack sampler ===
# pip install austin-python

# austin python script.py

# Ultra lightweight
# Production-ready


[OK] OPTIMISATION DATABASE

import sqlite3
import time

# === Batch Inserts ===

# [X] Lent - commit à chaque insert
conn = sqlite3.connect(':memory:')
c = conn.cursor()
c.execute('CREATE TABLE test (id INTEGER, value TEXT)')

start = time.time()
for i in range(1000):
    c.execute('INSERT INTO test VALUES (?, ?)', (i, f'value{i}'))
    conn.commit()  # Lent!
slow_time = time.time() - start

# [OK] Rapide - batch commit
conn = sqlite3.connect(':memory:')
c = conn.cursor()
c.execute('CREATE TABLE test (id INTEGER, value TEXT)')

start = time.time()
for i in range(1000):
    c.execute('INSERT INTO test VALUES (?, ?)', (i, f'value{i}'))
conn.commit()  # Une seule fois
fast_time = time.time() - start

print(f"Slow: {slow_time:.2f}s, Fast: {fast_time:.2f}s")
print(f"Speedup: {slow_time/fast_time:.0f}x")


# === executemany() ===

# [OK] Encore plus rapide
data = [(i, f'value{i}') for i in range(1000)]
c.executemany('INSERT INTO test VALUES (?, ?)', data)
conn.commit()


# === Connection Pooling ===

# Pour applications avec connexions multiples
from queue import Queue
import threading

class ConnectionPool:
    def __init__(self, database, size=5):
        self.size = size
        self.pool = Queue(maxsize=size)
        for _ in range(size):
            conn = sqlite3.connect(database, check_same_thread=False)
            self.pool.put(conn)
    
    def get_connection(self):
        return self.pool.get()
    
    def return_connection(self, conn):
        self.pool.put(conn)


# === Index pour queries rapides ===

# Sans index - O(n)
c.execute('SELECT * FROM test WHERE value = ?', ('value500',))

# Avec index - O(log n)
c.execute('CREATE INDEX idx_value ON test(value)')
c.execute('SELECT * FROM test WHERE value = ?', ('value500',))


[OK] BONNES PRATIQUES GÉNÉRALES

"""
1. MESURER AVANT D'OPTIMISER
   - Profiler code
   - Identifier bottlenecks
   - Optimiser hot paths seulement

2. CHOISIR BONNE STRUCTURE DE DONNÉES
   - list: Accès séquentiel, append
   - set: Membership testing, unique
   - dict: Key-value lookup
   - deque: Queue operations
   - array: Nombres homogènes

3. UTILISER BUILT-INS
   - sum() au lieu de loop
   - any()/all() pour conditions
   - min()/max() pour extrema
   - sorted() pour tri

4. ÉVITER COPIES INUTILES
   - Utiliser vues NumPy
   - Générateurs au lieu de listes
   - Modification in-place

5. PARALLÉLISER INTELLIGEMMENT
   - CPU-bound: multiprocessing
   - I/O-bound: asyncio ou threading
   - Considérer overhead

6. CACHE CALCULS RÉPÉTITIFS
   - lru_cache pour fonctions pures
   - cached_property pour attributs
   - Variables locales pour lookups

7. OPTIMISER HOT LOOPS
   - Sortir calculs constants
   - Variables locales
   - Vectoriser si possible

8. UTILISER OUTILS APPROPRIÉS
   - NumPy pour calculs
   - Numba pour loops Python
   - Cython pour performance critique

9. MAINTENIR LISIBILITÉ
   - Code lisible > micro-optimisations
   - Commenter optimisations non-évidentes
   - Mesurer impact réel

10. OPTIMISER PROGRESSIVEMENT
    - Commencer simple
    - Profiler
    - Optimiser bottleneck
    - Re-profiler
    - Repeat
"""


[OK] CHECKLIST OPTIMISATION

"""
AVANT D'OPTIMISER:
[WHITE_SQUARE] Profiler pour identifier bottlenecks
[WHITE_SQUARE] Établir benchmarks baseline
[WHITE_SQUARE] Définir objectifs performance

OPTIMISATIONS RAPIDES:
[WHITE_SQUARE] Utiliser built-in functions
[WHITE_SQUARE] List comprehensions au lieu de loops
[WHITE_SQUARE] join() pour concatenation strings
[WHITE_SQUARE] Sets pour membership testing
[WHITE_SQUARE] Local variables pour lookups répétés
[WHITE_SQUARE] Generator expressions pour grande data

OPTIMISATIONS STRUCTURES DONNÉES:
[WHITE_SQUARE] dict/set au lieu de list pour lookups
[WHITE_SQUARE] deque pour queues
[WHITE_SQUARE] array pour nombres homogènes
[WHITE_SQUARE] __slots__ pour classes nombreuses
[WHITE_SQUARE] defaultdict/Counter si approprié

OPTIMISATIONS CALCULS:
[WHITE_SQUARE] lru_cache pour fonctions pures
[WHITE_SQUARE] NumPy pour vectorisation
[WHITE_SQUARE] Numba pour loops critiques
[WHITE_SQUARE] Éviter nested loops

OPTIMISATIONS PARALLÉLISME:
[WHITE_SQUARE] multiprocessing pour CPU-bound
[WHITE_SQUARE] asyncio pour I/O-bound
[WHITE_SQUARE] ThreadPoolExecutor pour I/O simple
[WHITE_SQUARE] ProcessPoolExecutor pour CPU simple

APRÈS OPTIMISATION:
[WHITE_SQUARE] Re-profiler pour mesurer gains
[WHITE_SQUARE] Vérifier correctness (tests)
[WHITE_SQUARE] Documenter changements
[WHITE_SQUARE] Valider en conditions réelles
"""


[OK] ANTI-PATTERNS PERFORMANCE

# === 1. String Concatenation in Loop ===
# [X] TRÈS LENT - O(n²)
result = ""
for i in range(10000):
    result += str(i)  # Crée nouveau string à chaque fois!

# [OK] RAPIDE - O(n)
parts = []
for i in range(10000):
    parts.append(str(i))
result = "".join(parts)

# [OK] ENCORE MIEUX
result = "".join(str(i) for i in range(10000))


# === 2. Using + to Concatenate Lists ===
# [X] Lent - crée nouvelle liste
big_list = []
for chunk in chunks:
    big_list = big_list + chunk  # O(n) à chaque fois!

# [OK] Rapide - extend in-place
big_list = []
for chunk in chunks:
    big_list.extend(chunk)  # O(k) où k = len(chunk)


# === 3. Repeatedly Accessing Attribute ===
# [X] Lent
for i in range(10000):
    result = obj.attr.method(i)  # Lookup à chaque fois

# [OK] Rapide
method = obj.attr.method
for i in range(10000):
    result = method(i)


# === 4. Using List When Set Would Be Better ===
# [X] Lent - O(n) per lookup
items = [1, 2, 3, 4, 5, ...]
if target in items:  # Scan linéaire
    pass

# [OK] Rapide - O(1) per lookup
items = {1, 2, 3, 4, 5, ...}
if target in items:  # Hash lookup
    pass


# === 5. Not Using enumerate() ===
# [X] Moins lisible et légèrement plus lent
for i in range(len(items)):
    item = items[i]
    process(i, item)

# [OK] Plus pythonic et rapide
for i, item in enumerate(items):
    process(i, item)


# === 6. Using keys() Unnecessarily ===
# [X] Inutile
for key in my_dict.keys():
    value = my_dict[key]

# [OK] Direct
for key in my_dict:
    value = my_dict[key]

# [OK] Encore mieux
for key, value in my_dict.items():
    process(key, value)


# === 7. List Comprehension When Not Needed ===
# [X] Crée liste inutile
sum([x**2 for x in range(1000000)])

# [OK] Generator expression
sum(x**2 for x in range(1000000))


# === 8. Using append() in Nested Loop ===
# [X] Très lent
result = []
for i in range(100):
    for j in range(100):
        result.append((i, j))

# [OK] List comprehension
result = [(i, j) for i in range(100) for j in range(100)]


# === 9. Not Closing Files ===
# [X] Resource leak
f = open('file.txt')
data = f.read()
# Oubli de close()

# [OK] Avec context manager
with open('file.txt') as f:
    data = f.read()
# Fermé automatiquement


# === 10. Premature Optimization ===
# [X] Code illisible pour gain minime
result = [x for x in (y**2 for y in range(1000) if y % 2 == 0) if x > 100]

# [OK] Lisible d'abord, optimiser si nécessaire
squares = [y**2 for y in range(1000) if y % 2 == 0]
result = [x for x in squares if x > 100]


[OK] COMPARAISONS DÉTAILLÉES

# === String Concatenation ===
import timeit

def test_plus():
    s = ""
    for i in range(1000):
        s += str(i)
    return s

def test_join():
    return "".join(str(i) for i in range(1000))

def test_format():
    return "".join([str(i) for i in range(1000)])

print("String concatenation (1000 items):")
print(f"  +       : {timeit.timeit(test_plus, number=1000):.4f}s")
print(f"  join    : {timeit.timeit(test_join, number=1000):.4f}s")
print(f"  format  : {timeit.timeit(test_format, number=1000):.4f}s")


# === List vs Set Membership ===
def test_list_membership(items, targets):
    return sum(1 for t in targets if t in items)

def test_set_membership(items, targets):
    item_set = set(items)
    return sum(1 for t in targets if t in item_set)

items = list(range(10000))
targets = list(range(5000, 5100))

print("\nMembership testing (10000 items, 100 lookups):")
print(f"  List: {timeit.timeit(lambda: test_list_membership(items, targets), number=100):.4f}s")
print(f"  Set : {timeit.timeit(lambda: test_set_membership(items, targets), number=100):.4f}s")


# === Loop vs Comprehension ===
def test_loop():
    result = []
    for i in range(1000):
        result.append(i**2)
    return result

def test_comprehension():
    return [i**2 for i in range(1000)]

def test_map():
    return list(map(lambda x: x**2, range(1000)))

print("\nList creation (1000 items):")
print(f"  Loop   : {timeit.timeit(test_loop, number=10000):.4f}s")
print(f"  Comp   : {timeit.timeit(test_comprehension, number=10000):.4f}s")
print(f"  Map    : {timeit.timeit(test_map, number=10000):.4f}s")


# === Dict Access ===
def test_get():
    d = {i: i**2 for i in range(1000)}
    total = 0
    for i in range(1000):
        total += d.get(i, 0)
    return total

def test_bracket():
    d = {i: i**2 for i in range(1000)}
    total = 0
    for i in range(1000):
        total += d[i]
    return total

def test_try_except():
    d = {i: i**2 for i in range(1000)}
    total = 0
    for i in range(1000):
        try:
            total += d[i]
        except KeyError:
            total += 0
    return total

print("\nDict access (1000 lookups, all keys exist):")
print(f"  get()     : {timeit.timeit(test_get, number=1000):.4f}s")
print(f"  []        : {timeit.timeit(test_bracket, number=1000):.4f}s")
print(f"  try/except: {timeit.timeit(test_try_except, number=1000):.4f}s")


[OK] PROFILING EXEMPLE COMPLET

"""
Exemple: Optimisation d'un programme de traitement de données
"""

# === Version 1: Non optimisée ===
def process_data_v1(filename):
    """Version initiale - lente"""
    results = []
    with open(filename) as f:
        for line in f:
            parts = line.strip().split(',')
            if len(parts) == 3:
                name = parts[0]
                value = int(parts[1])
                category = parts[2]
                
                # Traitement
                if value > 100:
                    result = {
                        'name': name,
                        'value': value * 2,
                        'category': category.upper()
                    }
                    results.append(result)
    
    return results


# === Version 2: Avec profiling ===
import cProfile
import pstats

def profile_function(func, *args):
    profiler = cProfile.Profile()
    profiler.enable()
    result = func(*args)
    profiler.disable()
    
    stats = pstats.Stats(profiler)
    stats.sort_stats('cumulative')
    stats.print_stats(10)
    
    return result

# Profiler
# profile_function(process_data_v1, 'data.csv')

"""
Résultats profiling montrent:
- 40% temps dans strip()/split()
- 30% temps dans upper()
- 20% temps dans dict creation
- 10% temps dans append()
"""


# === Version 3: Optimisée ===
def process_data_v2(filename):
    """Version optimisée"""
    results = []
    append = results.append  # Cache method
    
    with open(filename) as f:
        for line in f:
            parts = line.strip().split(',')
            if len(parts) == 3:
                name, value_str, category = parts
                value = int(value_str)
                
                if value > 100:
                    # Évite dict creation si possible
                    append({
                        'name': name,
                        'value': value << 1,  # bit shift pour *2
                        'category': category.upper()
                    })
    
    return results


# === Version 4: Avec générateurs ===
def process_data_v3(filename):
    """Version avec générateur - économie mémoire"""
    with open(filename) as f:
        for line in f:
            parts = line.strip().split(',')
            if len(parts) == 3:
                name, value_str, category = parts
                value = int(value_str)
                
                if value > 100:
                    yield {
                        'name': name,
                        'value': value << 1,
                        'category': category.upper()
                    }


# === Version 5: Avec NumPy (si données numériques) ===
import numpy as np
import pandas as pd

def process_data_v4(filename):
    """Version vectorisée avec pandas"""
    # Lecture efficace
    df = pd.read_csv(filename, names=['name', 'value', 'category'])
    
    # Filtrage vectorisé
    df = df[df['value'] > 100]
    
    # Transformation vectorisée
    df['value'] = df['value'] * 2
    df['category'] = df['category'].str.upper()
    
    return df.to_dict('records')


# === Benchmark toutes versions ===
def benchmark_all(filename, runs=10):
    """Compare toutes versions"""
    import time
    
    versions = [
        ('V1: Original', process_data_v1),
        ('V2: Optimized', process_data_v2),
        ('V3: Generator', lambda f: list(process_data_v3(f))),
        ('V4: Pandas', process_data_v4),
    ]
    
    results = {}
    
    for name, func in versions:
        times = []
        for _ in range(runs):
            start = time.perf_counter()
            func(filename)
            times.append(time.perf_counter() - start)
        
        avg_time = sum(times) / len(times)
        results[name] = avg_time
        print(f"{name}: {avg_time:.4f}s")
    
    # Speedups
    baseline = results['V1: Original']
    print("\nSpeedups:")
    for name, time in results.items():
        if name != 'V1: Original':
            speedup = baseline / time
            print(f"  {name}: {speedup:.2f}x")


[OK] OPTIMISATION PAR CAS D'USAGE

# === Cas 1: Traitement de logs ===
def process_logs_slow(log_file):
    """Version lente"""
    errors = []
    with open(log_file) as f:
        for line in f:
            if 'ERROR' in line:
                parts = line.split()
                timestamp = parts[0]
                message = ' '.join(parts[2:])
                errors.append({'time': timestamp, 'msg': message})
    return errors

def process_logs_fast(log_file):
    """Version rapide"""
    errors = []
    append = errors.append
    
    with open(log_file) as f:
        for line in f:
            if 'ERROR' in line:
                # Split max 3 fois
                parts = line.split(maxsplit=2)
                if len(parts) >= 3:
                    append({'time': parts[0], 'msg': parts[2]})
    
    return errors


# === Cas 2: Traitement JSON ===
import json

def process_json_slow(data_list):
    """Parse JSON multiple fois"""
    results = []
    for item in data_list:
        parsed = json.loads(item)
        if parsed['value'] > 100:
            results.append(parsed)
    return results

def process_json_fast(data_list):
    """Parse une fois, filtre avec générateur"""
    parsed_items = (json.loads(item) for item in data_list)
    return [p for p in parsed_items if p['value'] > 100]


# === Cas 3: Calcul statistiques ===
def compute_stats_slow(numbers):
    """Parcourt liste plusieurs fois"""
    total = sum(numbers)
    count = len(numbers)
    mean = total / count
    
    variance = sum((x - mean) ** 2 for x in numbers) / count
    std_dev = variance ** 0.5
    
    minimum = min(numbers)
    maximum = max(numbers)
    
    return {
        'mean': mean,
        'std': std_dev,
        'min': minimum,
        'max': maximum
    }

def compute_stats_fast(numbers):
    """Un seul parcours"""
    if not numbers:
        return None
    
    # Calculs en un parcours
    total = 0
    total_sq = 0
    minimum = float('inf')
    maximum = float('-inf')
    count = 0
    
    for num in numbers:
        total += num
        total_sq += num * num
        minimum = min(minimum, num)
        maximum = max(maximum, num)
        count += 1
    
    mean = total / count
    variance = (total_sq / count) - (mean * mean)
    
    return {
        'mean': mean,
        'std': variance ** 0.5,
        'min': minimum,
        'max': maximum
    }


# === Cas 4: Cache HTTP responses ===
from functools import lru_cache
import hashlib

def make_hashable(obj):
    """Convertit dict en tuple hashable"""
    if isinstance(obj, dict):
        return tuple(sorted(obj.items()))
    return obj

@lru_cache(maxsize=128)
def cached_api_call(url, params_tuple):
    """Cache responses API"""
    import requests
    params = dict(params_tuple)
    response = requests.get(url, params=params)
    return response.json()

# Utilisation
def get_data(url, params):
    params_tuple = make_hashable(params)
    return cached_api_call(url, params_tuple)


# === Cas 5: Batch processing ===
def process_items_slow(items):
    """Traite un par un"""
    results = []
    for item in items:
        result = expensive_operation(item)
        results.append(result)
    return results

def process_items_fast(items, batch_size=100):
    """Traite par batches"""
    from multiprocessing import Pool
    
    def batch_processor(batch):
        return [expensive_operation(item) for item in batch]
    
    # Créer batches
    batches = [items[i:i+batch_size] 
               for i in range(0, len(items), batch_size)]
    
    # Traiter en parallèle
    with Pool() as pool:
        batch_results = pool.map(batch_processor, batches)
    
    # Aplatir résultats
    return [item for batch in batch_results for item in batch]


[OK] MONITORING PERFORMANCE EN PRODUCTION

# === Décorateur de monitoring ===
import time
import functools
from collections import defaultdict

class PerformanceMonitor:
    """Monitor performance de fonctions"""
    def __init__(self):
        self.stats = defaultdict(lambda: {
            'calls': 0,
            'total_time': 0,
            'min_time': float('inf'),
            'max_time': 0
        })
    
    def monitor(self, func):
        """Décorateur de monitoring"""
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            start = time.perf_counter()
            result = func(*args, **kwargs)
            elapsed = time.perf_counter() - start
            
            # Update stats
            stats = self.stats[func.__name__]
            stats['calls'] += 1
            stats['total_time'] += elapsed
            stats['min_time'] = min(stats['min_time'], elapsed)
            stats['max_time'] = max(stats['max_time'], elapsed)
            
            return result
        return wrapper
    
    def report(self):
        """Génère rapport"""
        print("\nPerformance Report:")
        print("-" * 70)
        print(f"{'Function':<20} {'Calls':>8} {'Avg':>10} {'Min':>10} {'Max':>10}")
        print("-" * 70)
        
        for func_name, stats in sorted(self.stats.items()):
            avg_time = stats['total_time'] / stats['calls']
            print(f"{func_name:<20} {stats['calls']:>8} "
                  f"{avg_time:>10.6f} {stats['min_time']:>10.6f} "
                  f"{stats['max_time']:>10.6f}")

# Utilisation
monitor = PerformanceMonitor()

@monitor.monitor
def process_data(data):
    time.sleep(0.1)  # Simule travail
    return len(data)

@monitor.monitor
def analyze_results(results):
    time.sleep(0.05)
    return sum(results)

# Utiliser fonctions...
for _ in range(10):
    data = list(range(100))
    result = process_data(data)
    analyze_results([result])

# Afficher rapport
monitor.report()


# === Logging performance ===
import logging
from contextlib import contextmanager

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

@contextmanager
def log_performance(operation_name):
    """Context manager pour logger performance"""
    start = time.perf_counter()
    try:
        yield
    finally:
        elapsed = time.perf_counter() - start
        logger.info(f"{operation_name} completed in {elapsed:.4f}s")

# Utilisation
with log_performance("Data processing"):
    # Code à mesurer
    process_large_dataset()


# === Alerting sur performance ===
class PerformanceAlert:
    """Alerte si performance dégradée"""
    def __init__(self, threshold_seconds):
        self.threshold = threshold_seconds
    
    def __call__(self, func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            start = time.perf_counter()
            result = func(*args, **kwargs)
            elapsed = time.perf_counter() - start
            
            if elapsed > self.threshold:
                logger.warning(
                    f"{func.__name__} took {elapsed:.4f}s "
                    f"(threshold: {self.threshold}s)"
                )
            
            return result
        return wrapper

@PerformanceAlert(threshold_seconds=1.0)
def slow_function():
    time.sleep(1.5)


[OK] TIPS & TRICKS

# === 1. Use dis to understand bytecode ===
import dis

def example():
    x = 1
    y = 2
    return x + y

dis.dis(example)
# Voir bytecode Python pour comprendre performance


# === 2. timeit magic in Jupyter ===
"""
%timeit sum(range(1000))
%%timeit
total = 0
for i in range(1000):
    total += i
"""


# === 3. Use __debug__ for debug code ===
if __debug__:
    # Code debug seulement
    print("Debug mode")

# Désactiver avec: python -O script.py


# === 4. Profile imports ===
# python -X importtime script.py
# Montre temps d'import de chaque module


# === 5. Use sys.getsizeof carefully ===
import sys

# Taille shallow (pas récursif)
size = sys.getsizeof([1, 2, 3])

# Pour taille totale, utiliser pympler
# from pympler import asizeof
# size = asizeof.asizeof([1, 2, 3])


# === 6. Benchmark avec perfplot ===
# pip install perfplot
"""
import perfplot

perfplot.show(
    setup=lambda n: list(range(n)),
    kernels=[
        lambda lst: [x**2 for x in lst],
        lambda lst: list(map(lambda x: x**2, lst)),
    ],
    labels=["comprehension", "map"],
    n_range=[2**k for k in range(15)],
)
"""


# === 7. Use __future__ imports ===
from __future__ import annotations  # PEP 563 - Postponed annotations
# Réduit overhead des type hints


# === 8. Profile memory avec tracemalloc ===
import tracemalloc

tracemalloc.start()

# Code à profiler
data = [x**2 for x in range(10000)]

current, peak = tracemalloc.get_traced_memory()
print(f"Current: {current / 10**6:.2f}MB, Peak: {peak / 10**6:.2f}MB")

tracemalloc.stop()


# === 9. Use weakref pour caches ===
import weakref

class Cache:
    """Cache avec weak references"""
    def __init__(self):
        self._cache = weakref.WeakValueDictionary()
    
    def get(self, key):
        return self._cache.get(key)
    
    def set(self, key, value):
        self._cache[key] = value


# === 10. Optimize imports ===
# [X] Import tout
import numpy

# [OK] Import seulement ce dont vous avez besoin
from numpy import array, sum

# [OK] Pour packages lourds, importer dans fonction
def heavy_computation():
    import tensorflow as tf  # Import local
    # ...


[OK] RESSOURCES ET OUTILS

"""
PROFILING TOOLS:
- cProfile: Built-in, détaillé
- line_profiler: Par ligne
- py-spy: Sampling, production-ready
- scalene: CPU + Memory + GPU
- viztracer: Visualisation timeline

MEMORY PROFILING:
- memory_profiler: Ligne par ligne
- tracemalloc: Built-in
- memray: Moderne, rapide
- pympler: Analyse objets

BENCHMARKING:
- timeit: Built-in, précis
- perfplot: Graphiques performance
- pytest-benchmark: Pour tests

OPTIMIZATION LIBRARIES:
- NumPy: Calcul vectorisé
- Numba: JIT compilation
- Cython: Compilation C
- PyPy: JIT Python

ASYNC/PARALLEL:
- asyncio: I/O async
- aiohttp: HTTP async
- multiprocessing: CPU parallel
- concurrent.futures: Threading/Process pool

MONITORING:
- psutil: System resources
- resource: Process resources
- gc: Garbage collector stats

DOCUMENTATION:
- Python Performance Tips: wiki.python.org/moin/PythonSpeed
- Real Python Performance: realpython.com/python-performance
- NumPy Performance: numpy.org/doc/stable/user/performance.html
- Numba Docs: numba.pydata.org
"""


[OK] RÉCAPITULATIF FINAL

"""
GOLDEN RULES:
1. Measure before optimizing (profiling!)
2. Optimize bottlenecks, not everything
3. Use right data structures
4. Leverage built-ins and libraries
5. Consider readability vs performance
6. Test after optimization

QUICK WINS:
- List comprehensions
- join() for strings
- Sets for membership
- Local variables
- Built-in functions
- Generators for large data

FOR CPU-BOUND:
- NumPy vectorization
- Numba JIT
- Multiprocessing
- Cython for extreme cases

FOR I/O-BOUND:
- asyncio
- Threading
- Connection pooling
- Batch operations

FOR MEMORY:
- Generators
- Slots
- Numpy arrays
- Weakref caches

REMEMBER:
"Premature optimization is the root of all evil" - Donald Knuth
Profile -> Optimize -> Measure -> Repeat
"""