Dealing with sensitive material as a group
Certain topics may be offensive, uncomfortable, emotionally difficult, or just plain not fun for certain players. The best way to handle "mature" subject matter is to get in the habit of talking to your group members about your play experience in an honest and straightforward fashion.
Establish some limits ahead of time. The character/campaign creation session is a great time to talk about the big issues that are likely to come up in play. Just like you should all know what kind of playstyle to expect from the game, you should also figure out some basic expectations about its subject matter and how you're going to handle sensitive topics.
More importantly, though, you should be ready and willing to communicate about issues as they arise in play. In particular, be attentive to your friends' reactions; you can usually tell when someone has a problem with what's going on in the game even when they don't articulate it right away.
I recommend a technique called "lines & veils," which you can use to help you think about and talk about how you handle sensitive subjects. Basically:
A line is, well, a line — a hard limit, something we do not want to cross. Lines represent places we don't want to go in roleplaying. Our lines define the limits that we all agree to respect.
A veil is a "pan away" or "fade to black" moment. When we veil something, we're making it a part of the story, but keeping it out of the spotlight. Think of it as a way to still deal with certain themes while avoiding having to describe them in graphic detail.
See our lines-and-veils topic for more information.
Since you mentioned Game of Thrones as an example, I think it's worth pointing out that you can have this discussion about more than just sex — you probably want to address how your game will deal with violence, torture, rape, misogyny, and slavery, for example.
Now, none of this is GM-centric advice. The GM can take the lead in initiating this discussion, but it's about what everybody wants and is comfortable with. All of the players at the table are responsible for each other's fun.
Sexuality in games
How you represent sex is about more than just defining what you're comfortable with at the table, though. There's also the question of what role sex and sexuality play in the fiction as a whole.
For example:
In romantic comedies and light romantic dramas, sex is often a reflection of the characters' relationship hitting a particular stage (sometimes an intermediate "getting to know each other better," sometimes the end goal of a stable and satisfying relationship). Characters tend to have destined partners — it's a foregone conclusion who'll end up together, we're just playing out the challenges they face along the way.
- Adventure stories often use the same approach, just contained within a side plot that develops concurrently with the main procedural plot.
Tragedies and romantic dramas are similar in many ways. They also tend to use sex to indicate the growth of a relationship — but now that growth can have all kinds of sorrowful or conflicted dimensions. They tend to also have "destined" pairings, but put them under serious threat; you really don't know whether everything will work out or not. Many romantic dramas are about discovering loving sexuality as a way to deal with some trauma in a character's past.
Going "darker" without using the standard romantic-drama template lets you explore sides of sexuality that don't fit into conventional story framework. Often you'll see this in dark comedies like Secretary or Harold and Maude — comedic absurdity allows you to talk frankly about sex and desire, while the dark tone makes it okay to delve into "heavy" or unusual stuff, the parts of sex and love that we often feel conflicted about as a culture.
Horror fiction tends to play up the transgressive and dangerous aspects of sexuality. A lot of monsters have a sexual dimension to them. Sometimes this makes them captivating tempter (vampires, like, everywhere). Other times, it makes the monsters seem feral and uncontrollable in a familiar human way (Ginger Snaps, for instance). Often it's both together!
- Of course, part of sex being dangerous is that sometimes it does just actually destroy you; some horror makes this particularly egregious by punishing all of the characters for their sexual availability until only the pure "final girl" is left. I'd find it hard to play out this one in an RPG as anything other than genre-aware slapstick comedy, to be honest.
When you go full-on "dark and gritty," there's a tendency for sex to be represented kinda shallowly: it's exploitative, it's aggressive, it's a perk of power, it's the product of selfish desires, the end. That's because deeply pessimistic fiction is about as constrained as deeply optimistic fiction, when you come down to it — keeping a very consistent tone tends to wash away nuance a bit. (Be wary of doing this at the table because it tends to lead to pervasive emotional abuse and rape.)
Generally the most nuanced portrayals of relationships are in works that don't really focus so much on how they begin. Instead, you see a fully-developed relationship, with deep connections between the characters, that's evolving over time. When you free yourself from the need to laboriously justify and establish deep connections between the characters, you can explore those connections with greater depth and variety. Some examples of this in action are Amour, Sid and Nancy, or The Americans. Scandinavian LARPers have a "six months rule" that's all about making this happen in play (not just for romantic relationships but in general):
Anyone in a scene with you, you had known for at least six months. The basic principle behind this is that it's much more interesting to see the middle of a relationship than its beginning.
So, as you can see, you've got lots and lots of options for what you do with sexuality in play! One good way to talk about this is to reference books, films, and shows that have a similar tone to what you want to see in play. (Make sure people have actually experienced the works in question, though — sometimes folks think they know what something is like from hearsay, but it turns out to be entirely different; the "texture" of a work is harder to communicate than its overall plot and theme.)
Be mindful of the difference between other media and RPGs, though. Nearly all commercial media is, well, mass media. Even if it's targeting a niche audience it has to appeal to a fairly broad audience. Whereas what happens in an RPG is just for you and your friends at the table. You've got tons of leeway to approach things from a novel angle or depict aspects of sex and relationships that you don't see much of on TV.
Sexual violence
Particularly with "darker" portrayals of sexuality, you can end up with situations where sex is mixed with coercion or violence — rape, assault, domestic abuse, &c. This is, like, why lines and veils exist, right here.
Please never assume that just because sex happens in your game, rape should, too. (And never, ever, EVER assume that just because female characters are present in a game, rape should be, too.) It's totally reasonable to have stories that feature even deeply problematic sexuality — self-destructive sex, emotionally tense dysfunctional relationships, manipulative seduction, &c. — without ever mentioning rape.
If your group's game is going to incorporate sexual violence or the threat of sexual violence, spend some time talking about how and why. Also, be mindful of the fact that this is one of the "lines" where people are most likely to change their mind in the moment — you think "Oh, yeah, that'll be fine," but then you experience it in game and decide "No, I want to stop." If that happens at the table, then do just stop.
Because sexual violence in an RPG can be so uncomfortable, I recommend avoiding the theme altogether unless you as a group have something serious to say about it. That means going beyond the tropes you encounter in media (for example, in reality, rape isn't a thing that just happens to women).
Some practical concerns
These are some basic practical points that might come up, especially in D&D.
Sex and "race:" Fantasy RPGs often feature a variety of human-like species ("races"), some of whom are sexually compatible enough to produce offspring. They also tend to feature unfortunate subtext, like the implication that half-orcs are the children of rape, or fantasy genetics that's basically a literal version of the "one-drop rule." I lean towards excising this stuff when possible.
STIs and pregnancy: In the real world, sex carries the risk of sexually-transmitted infections, as well as unintended pregnancy. But, in the real world, getting stabbed through the side with a longsword is much more unpleasant than it proves to be in many RPGs. So, if the annoying or troublesome aspects of sex don't add anything to the game, it's okay to ignore them. I promise you it won't ruin anything if characters can only get pregnant on purpose. You could justify it by saying they have access to herbal contraception, or contraceptive magic, or just say it's a story thing (like how you never see anyone poop in dungeons) and not justify it at all. Likewise, it's usually more fun to merely joke about a character getting an STI than to actually give them one.
Sex and magic: Whether your fantasy setting depicts magic as something similar to the magic of historical real-world belief systems or something more akin to an alternative kind of technology, it makes sense that people would use magic for sexual pleasure and procreation. I don't think trying to integrate sex-themed spells into D&D's existing framework will give you a satisfying result, just like there's not really a particularly satisfying mechanic for using D&D magic to make crops grow or divine the future. So, in general, don't worry about it. You don't need detailed rules for every single thing anyone ever does in the setting, especially if it's an activity reserved for NPCs or mainly background stuff.
Sex magic: Another theme is straight-up sex magic: magic that's powered and mediated by sex. Really this is all about the symbolic power of sex, so I'd approach this the same way I'd approach defining the role of sex in your game in general.
Fan works like the Book of Erotic Fantasy try to cover these topics, but generally they strike out because they don't lay down the core thematic groundwork first. So everything just kinda comes off as a sex joke by default — and shoehorning a sex joke into D&D mechanics just renders it overlong and unfunny.
Game-mechanical inspiration
If you're looking for game-mechanical bits to draw on, consider some of these:
Sex and intimacy: Apocalypse World and Monsterhearts have "sex moves," which are mechanics that trigger when you have sex. These are used to reflect the effects of personal intimacy upon the characters. The idea is that, narratively, sex is a concrete reflection of the connection between two characters. You can react positively or negatively afterward, you can have sex with your closest friend or a stranger, but what you can't do is have sex that doesn't matter (unless you are the battlebabe, who is too cool to be affected by sex).
Romantic attraction: Breaking the Ice is a full game about romantic interaction between two people. The game has mechanics modeling attraction and compatibility, but its real focus is using those mechanics to encourage you to talk about the characters and how they relate to each other. It's also got some good discussion about the different kinds of love stories and how they can fit into a larger work.
Marriage and children: Pendragon is a game about Arthurian knights, designed for long-arc play where the characters manage their estates and raise children between adventures. It's a good — albeit gender-essentialist — reference for ideas about getting married and growing a family. Sagas of the Icelanders is less structured but has a few moves you can use for inspiration as well.
