End-to-End (E2E) learning enables autonomous driving models to directly predict control commands from raw images, but their performance is strongly influenced by dataset bias. In this study, training data were collected using the LIMO platform with a front-facing camera and joystick control, revealing significant limitations such as excessive straight driving, biased turning, fixed illumination, and short repetitive tracks. These limitations restrict generalization to diverse driving scenarios. To address this issue without resorting to costly simulation-based methods, lightweight data augmentation strategies were applied, including ColorJitter, Gaussian blur, Grayscale Conversion. Two pretrained vision encoders, ResNet18 and PP-LCNet, were benchmarked with different model scales. The proposed approach achieved a reduction in trajectory error of up to 55% and improved attention to road-relevant regions, as confirmed by Grad-CAM analysis. This study demonstrates that simple augmentation techniques can effectively mitigate dataset bias and enhance the generalization capability of lightweight E2E driving models. The proposed method reduced driving trajectory error by up to 55% and shortened completion time by approximately 12%, demonstrating the effectiveness of lightweight augmentations.