The health secretary says he is concerned about the “poor conditions” on the streets of Birmingham.