Emotion-aware computing presents one of the main challenges in contemporary natural human interaction research, with natural expressive speech synthesis being one of its next significant milestones. This paper describes ongoing work on developing a generalized framework for annotating, analyzing and modeling expressive speech in a data driven approach, towards building expressive text to speech synthesis systems. To this end, the data driven methodology is described, comprised of the acoustic analysis and expression clustering together with an online environment for speech perception and annotation. Preliminary results are also given, providing encouraging indications for further research.