The rapid growth of wireless mobile broadband communication networks has fueled new capabilities in scalable device-to-edge-to-cloud continuum, ranging from increased data rates of 1~10 Gbps, ultra-low latencies of 1ms or less, larger coverage with massive number of devices connected 24x7. These advances have enabled exciting new edge native applications, such as Augmented Reality/Virtual Reality (AR/VR), video analytics, and 3D on-device intelligence. However, unlike Clouds, edge clients have little elasticity in computing and communication resources. Edge devices are intermittently connected to the Internet, inherently heterogeneous in computing resource, and more exposed to privacy and security violations. In this keynote, I will use on-device intelligence and distributed AI as two emerging and complimentary distributed learning paradigms in navigating this device-edge-cloud continuum, while considering resilience, privacy, and multi-tenancy of shared and heterogeneous resources. I will describe alternative distributed learning architectures and optimization strategies, enabling edge system adaptability and robustness, while preserving good application fidelity (level of accuracy).