BOSTON, Aug. 25— Representative Barney Frank of Massachusetts said today that without his knowledge a male prostitute he had hired to do personal errands had run a prostitution service from the Congressman's Washington apartment.
Mr. Frank, who made his homosexuality public in May 1987, said he had paid for sex with the prostitute one time, in the spring of 1985, and had hired him as a personal employee later that year because he thought he could encourage the man to ''change his life.''
''Thinking I was going to be Henry Higgins and trying to turn him into Pygmalion was the biggest mistake I've made,'' Mr. Frank said at a news conference in Newton, a Boston suburb that is part of the state's Fourth Congressional District. Mr. Frank, a 49-year-old Democrat, has represented the district since 1981. 'A Very Good Con Man'
Conceding ''poor judgment,'' Mr. Frank said: ''It turns out that I was being suckered. He was, among other things, a very good con man.''
Mr. Frank's remarks followed publication of an article today in The Washington Times that disclosed the Congressman's links to a prostitute and the man's use of the Capitol Hill apartment for a prostitution service.
The newspaper article called the man by what it said was a ''professional name''; Mr. Frank said today his name was Steven Gobie. Washington police records show that Mr. Gobie has a history of offenses, including drug charges.
The Congressman said that to help Mr. Gobie, he hired him as a housekeeper and driver. Mr. Frank also said he made some calls and wrote letters, using Congressional stationery, to Mr. Gobie's probation officer in Virginia to let the office know he was employed. Proof of employment is usually a condition of probation. Knowledge Is Denied
In the Washington Times article, the man was quoted as saying that Mr. Frank knew of the use of his apartment for prostitution. At his news conference, Mr. Frank denied having such knowledge.
Efforts by The New York Times to reach Mr. Gobie at various listed addresses today were unsuccessful. But in an interview on Washington television station WUSA this evening, Mr. Gobie told of bringing ''clients'' to the Congressman's apartment and he acknowledged that he had been convicted on felony sex and drug charges in Virginia.
Mr. Frank said he dismissed Mr. Gobie in August 1987 after his landlords had told him for the second time about people coming and going from the apartment while he was away.
The Congressman said that he had paid Mr. Gobie with his own money for the personal errands and that Government or campaign money was not involved. Mr. Frank acknowledged that he broke the law by patronizing a prostitute but said he had not violated any ethics laws or Congressional rules.
The Congressman said the disclosure would not diminish his influence in Congress, where he is known for sharp wit and skillful floor debate and where he is considered a leading voice of liberalism.
Mr. Frank said the voters of his district would not judge him on the basis of indiscretions in his private life. On the other hand, Mr. Frank said the incident is likely to be used against him in a campaign. ''If I was planning strategy, this is not one of the events I would propose,'' he said. Movitivated by Criticisms
Mr. Frank said his actions were motivated in part by criticism that liberals ''are interested in helping humanity at large, maybe with a vote or with a check, but that we don't show a willingness to get involved in particular situations, or particular individuals who might need help.''
Although Congress is in recess and its members on vacation, there were indications that Republican lawmakers would seek an ethics investigation of Mr. Frank.
Edward Rollins, co-chairman of the National Republican Congressional Committee, said of The Washington Times article: ''The story speaks for itself. I would expect members of the Ethics Committee to ask for a full investigation.''
But Speaker Thomas S. Foley, Democrat of Washington, strongly supported Mr. Frank. ''There is no more able, articulate and effective member of the House of Representatives than Barney Frank,'' he said.
The Massachusetts Republican Party rejected Mr. Frank's explanation and called on him to resign.
''Frank has been on the front lines in railing against the 'sleaze factor,' which he attributes to the Republican Party,'' said Alexander Tennant, the party's executive director. ''It is hypocritical of him to say that in paying for sex he is guilty of nothing more than being a humanitarian social worker.''
In offering the support of the state Democratic party, its chairman, Representative Chester G. Atkins, described Mr. Frank as ''among the handful of the most gifted and talented legislators.''
''This is obviously a difficult personal time for Barney,'' he added, ''but his personal life has nothing to do with the way he performs his public duties.''
Representative Gerry E. Studds, the only other publicly gay member of Congress, has easily won re-election since he was censured by the House in 1983 for an affair with a 17-year-old page. Mr. Studds is also a Democrat from Massachusetts. Sex for Pay Acknowledged
Mr. Frank acknowledged that he paid for sex with Mr. Gobie, whom he described as 26 or 27 years old at the time, in the spring of 1985, after responding to an escort service advertisement in a local publication, although he was not certain that was how the two made contact. What began as a sexual and personal relationship gradually became a friendly one without sex, he said.
''After meeting him a couple of times, it occurred to me that he really was sufficiently intelligent and had sufficient personal skill that it was really a mistake for him to be doing this,'' Mr. Frank said.
''I said to him, 'I think you're kind of wasting your life.' ''
While he did write to Mr. Gobie's probation office, Mr. Frank said, he declined Mr. Gobie's request to use his political influence to try to reduce his probation.
''All I said to the probation department was, 'I will employ him so that he could apply to get his probation not extended,' '' Mr. Frank said.
The article in The Washington Times quoted Mr. Gobie as saying he needed Mr. Frank's letters to the probation department to be allowed to work outside Virginia.
Mr. Gobie's compensation totaled $20,000 a year: a combination of cash, payments for a lawyer and court-ordered sex therapy, and the value of the use of the Congressman's car and telephone, Mr. Frank said. Mr. Gobie lived elsewhere but occasionally stayed at the Congressman's apartment when Mr. Frank was traveling, he said. Prostitution Not Specified
The Congressman said he told Mr. Gobie that no drugs were permitted in his apartment, but it did not occur to him to say, ''Don't engage in prostitution.'' Mr. Frank said he knew that other people came to the apartment but he thought they were Mr. Gobie's friends.
Mr. Frank said it became clear by August 1987 that Mr. Gobie had not changed his ways after his landlords notified him a second time of unusual activity at the apartment that might have involved prostitution. At that point, Mr. Gobie was dismissed, Mr. Frank said.
The Congressman said Mr. Gobie telephoned in December 1987, saying he was having a hard time. Mr. Frank said he offered to pay Mr. Gobie to do errands. Among the jobs was picking up cleaning, and when Mr. Gobie refused to return the Congressman's clothing, Mr. Frank said he broke off the job.
Mr. Frank said the episode had become public because Mr. Gobie ''decided to get some revenge'' by telling his story to The Washington Times.
Mr. Gobie has a history of run-ins with the local police. Court records in Washington show that he was placed on four years' probation in 1975, when he was 18 years old, for selling and possessing narcotics. The Washington Times reported that he was convicted in Virginia in 1982 on two obscenity charges, one sodomy charge and a single charge of cocaine possession. He was on probation for those crimes at the time Mr. Frank hired him, the paper said.
After he left Mr. Frank's employment, in early 1988, Mr. Gobie was twice arrested by the police in Washington and charged with burglary and shoplifting food from a supermarket. He was not prosecuted on the first charge.
Mr. Frank disclosed his homosexuality in May 1987. He said today that one reason he did so was that entanglements like the one with Mr. Gobie made him ''vulnerable to being preyed on.'' He told The Associated Press late Thursday that he had been in a monogamous relationship for about two years with ''a very sensible person and a steadying influence.''
photo of Rep. Barney Frank (AP)