Manhattan Congressman Charles Rangel announced Wednesday that he will temporarily step down from his post as head of the powerful House Ways and Means Committee amid the latest ethics probe into his travels.

Watch the complete announcement.

The Harlem Democrat has been under fire ever since the House Ethics Committee released a report last week that found he had violated House rules by accepting two corporate-sponsored trips to the Caribbean.

Rangel said that he asked House Speaker Nancy Pelosi for a leave of absence from the position until a congressional ethics committee completes its investigation.

In the very brief news conference on Capitol Hill, Rangel said he is leaving "to avoid my colleagues having to defend me'' during the campaign season.

Speaking exclusively to NY1 afterwards, Rangel said he didn't want to harm fellow Democrats facing tough reelection fights or become a distraction for Speaker Pelosi.

"It's not fair to her and it's not fair to the many freshman and those who have close districts that instead of getting their message out they are asking about me," Rangel said.

The Harlem Democrat insisted his move is a temporary one, but Republicans disagree, arguing it would require a full vote of the House to reinstate him.

Veteran Congressman Pete Stark of California will serve as acting chairman, according to Colorado Representative John Salazar.

Stark is the next most senior Democrat on the panel.

NY1 spoke with some residents from Rangel's district in Harlem, and asked them what they thought of Wednesday's announcement.

"I think he has done a magnificent job and I think he will continue," said one voter.

"It does change my view. If he's stepping down then that means that he feels there's a problem, or he just doesn't want any more hassle for the Democrats," said another.

"They haven't proven anything, so why should he uncomfortable about it," said a third. "He's just stepping down temporarily; that's good enough."

Elected officials say that despite the recent controversies, he will always be remembered for his work for the state.

"I have not talked to Charlie yet. I will call him," said Mayor Michael Bloomberg. "He's done a lot for New York over the years and I certainly wish him the best."

"I think Charlie Rangel is a great leader for New York. He's someone who's fought for this country, who's fought for New York every step of the way," said Senator Kirsten Gillibrand. "I'll respect his decision, but I think it's a great loss for New York. We'll see how things continue."

On Tuesday, Rangel insisted that he was not giving up the chairmanship, despite the fact that he reportedly did not have enough votes from fellow lawmakers to hold onto the job.

An ethics panel is still investigating accusations regarding Rangel's fund raising, his failure to pay taxes on rental income from a villa he owns in the Dominican Republic, and his use of rent-stabilized apartments in Manhattan.