Credit rating for back taxes?

I've been living abroad for many years, and now I plan to return to the US. I have just filed tax returns for the past 3 years, and I'm in the process of paying some back taxes plus penalties. Will this hurt my credit rating? I plan to buy a house soon in the US, and I'm wondering how this will effect me, if at all. Thanks!