Modeling and recognizing the underlying dynamics of living entities at multiple spatio-temporal scales are challenging tasks. They provide a deeper understanding of dynamic processes related with detection, tracking, recognition, and characterization of behavior and occurrence of events. This talk will provide intelligent video systems from three domains: 1) video networks for human detection, tracking and recognition, 2) video bioinformatics for characterizing the health of cells and their growth and 3) video understanding of emotions for developing intelligent agents. It will present examples of new representations, indexing/matching and learning techniques for robust individual identification, re-identification, tracking; quality control of human embryonic stem cell colonies and applications in advertising.