So finally I came across another post and here is the correct setting I have done with.
The settings are as followed for manual setup for IMAP.
INCOMING SERVER SETTINGS
IMAP server - imap.mail.yahoo.com
Port - 143
Security type - TLS
IMA path prefix - (leave blank)
OUTGOING SERVER SETTINGS
SMTP Server- smtp.mail.yahoo.com
Port - 587
Security type - None
(check) require sign-in
Hope this helps. Comments if you find there are mistakes.