One question would be whether the homosexual-advocacy groups in question are the MAJOR sponsors, or if the general public is going to perceive this event as one centered around these groups.
Another question concerns what capacity you will be acting in as a speaker at this event; are you a religious? Are you representing a Catholic organization in any sense? Are you speaking as an expert on some point which can be understood differently by Catholics and non-Catholics (such as abortion, secular education, etc)?
Finally, there is the question of whether YOU personally can expect to serve this event with your speaking gifts and NOT be at least implicitly associated with the homosexual groups. Remember that our associations are not only possible occasions of sin for us, but for others who observe us as well. If people are led to believe that you have anything in common with these other groups other than speaking engagements at the same event, then it is incumbent upon you to separate yourself from the event and make sure you are not clothed with their immorality by association.
Ask yourself whether, granted their permissions or not, you would be able to address the topic in question from a faithful, traditional Catholic perspective, or to be able to distribute books or other literature as support for your position on the issue being presented.
As a person who played secular music for years in bars and nightclubs, I am even more impressed these days with the necessity of avoiding even the APPEARANCE of sin - I can only imagine what people thought of my character when they saw me entering or performing in some of the places I did.
My prayers will be for you to 'hold fast the traditions you have received' from the Apostles and not to shrink at representing Christ and His Church, whether by attendance at the event or by graciously declining.
St. Anthony of Padua, gifted preacher of the holy Gospel, pray for us.
Sacred Heart of Jesus, have mercy on us.