An efficient classification algorithm is proposed for characterizing breast lesions. The algorithm is based on the cubic least squares mapping and the linear-kernel support vector machine (SVMLSM) classifier. Ultrasound images of 154 confirmed lesions (59 benign and 52 malignant solid masses, 7 simple cysts, and 32 complicated cysts) were manually segmented by a physician using a custom developed software. Texture and outline features and the SVMLSM algorithm were used to design a hierarchical tree classification system. Classification accuracy was 98.7%, misdiagnosing 1 malignant an 1 benign solid lesions only. This system may be used as a second opinion tool to the radiologists.