My 21 yr old brother is living with us?

he is unemployed is always texting watching tv never cleans sleeps till noon and I have him taking care of my 2 yr old son until he finds a job. Its one of my sisters birthday today and he wants to go and with my other sister and take her a gift and a cake my problem is i dont like my son being out on the streets i feel better if he is at home taking care of him he wants to take him to the bd gathering am i exagerating should i let him take him. I yelled at him and told him that my sister should pick him up to take him to apply for a job not to go sit around at my other sisters house. what should i do?