I love Rogues on a conceptual level, and in other RPGs, I've often gravitated toward them as a kind of secondary archetype (or tertiary - I tend to like heavily armored melee/magic hybirds first and then some kind of spellcaster, often with a "dark" vibe second). In WoW, for example, my Undead Subtlety Rogue was the first I ever got to the then-level cap of 70 in Burning Crusade (19 years ago!) In D&D, at least in 5E, Rogues play a sort of weird role in a group. They're designed primarily as a utility class, but without the magical spells that a Wizard or Artificer brings to the table (though Arcane Tricksters, of course, lean back this way). Thanks to having more skill proficiencies, expertise, and eventually (and coming online earlier now than it did in 5.0) Reliable Talent, they're the "good at skills" class.
Personally, I don't think this really does the premise of the class justice. While it's certainly a cool thing to have, I A: don't think that this is necessarily the first thing you think of when you think of the Rogue as a fantasy archetype and B: seems to justify disempowering them in combat.
To be clear, the math on Rogue damage isn't all that straightforward. For one thing, Sneak Attack damage goes up every two levels, so there's less of a clear "breakpoint" in their damage output. This damage isn't necessarily consistent either: certain combat scenarios will take Sneak Attack off the table, such as if there's any source of disadvantage. Because it's such a giant chunk of a Rogue's damage, though, it also means that if we can find ways to make multiple attacks, this can boost our overall damage output significantly because as long as we hit with one of them, most of our damage is going through (I say most, but this doesn't really become the case until we get to around level 10 or 12).
I think that if we want to make the Rogue work in combat, we've got to make big changes to Sneak Attack - honestly, tearing the whole thing apart and building something with the same name might be the best option.
But, let's not get ahead of ourselves:
One thing I do think is worth preserving is that Rogues should be using lower-damage-die weapons. The classic archetype is a character who wields daggers and shortswords, and the Shortbow is a better fit for a sneaky Rogue (while Longbows are Heavy and thus can benefit from Great Weapon Master, I'm inclined to believe that without that, the fact that Shortbows have Vex make them usually the better weapon in 5.5).
Frankly, the lack of proficiency in higher-damage weapons is the only "price" I think Rogues should pay for getting some kind of source of additional damage. (Rapiers are actually toward the higher damage numbers, though I think probably most Rogues who are going for melee at all are probably dual-wielding a Shortsword and Scimitar).
Part of me wants to create an incentive to wield a dagger at least in the off-hand, but I think we'll just let the Thrown property do that for us (and maybe the Piercer feat).
Crucially, though, this means that the thing we're no longer taking away from the Rogue is that massive boost that all other martial characters (even some subclasses of not-typically-martial classes): Extra Attack.
Honestly, I think this will go a long way toward helping Rogues catch up. It's not enough to fully replace Sneak Attack, obviously, but it's a big start.
But what beyond that?
Well, now we need to address Sneak Attack.
In a vacuum, Sneak Attack adds 3.5 to 35 damage per turn over the course of your leveling journey. The actual number is a little more complicated because of the chance to hit and the chance to miss, as well as the chance that you're not actually going to get it. Actually calculating the chance you get Sneak Attack is kind of impossible, as all manner of scenarios might present themselves.
So, let's consider our first change:
You just get to choose to get Sneak Attack. No requirements to have advantage or an ally nearby, and you don't lose it if you had a source of disadvantage on the attack (as I interpret it, even if you cancel out disadvantage, if there's any source of disadvantage, you're SOL).
What impact would this have, then? I'm going to do a very quick-and-dirty version of my "vs Wyvern" calculations here: if we had a +9 to hit (not unreasonable for a level 10 Rogue) and were making two Shortbow attacks (we're ignoring Vex for brevity's sake) we'd be hitting on 5s or higher, getting an 80% chance to hit. Two attacks would mean 1d6+5, or 8.5, twice, which gives us 17 80% of the time and a crit bonus of 7 5% of the time (usually I do the attacks separately, but it's just a factor of two either way) so that's 13.6+.35, or 13.95. Then, our Sneak Attack at this level is 5d6, or 17.5 average. The chance we get it at all is 96% (the chance not to would mean the 20% miss chance happening twice) so that's 16.8, and then the chance we get a crit with it is 5% (chance on the first attack) or 5%x20% (chance we crit on the second attack after missing on the first) which is conveniently a 1% chance, so that's a total of 6%x17.5, or 1.05. Add it up and Sneak Attack is giving us 17.85, so when added to our normal damage, that's 31.8.
That's actually pretty decent damage, and it's even lowballing it because of the effect that Vex will have on it.
So, if that satisfies you, we can just kind of stop here: just say you can do Sneak Attack once per turn when you hit with an attack. And, of course, give Rogues Extra Attack.
But I actually think this could be something of an overcorrection.
One of the issues we run into, I think, is that Great Weapon Master casts such a long shadow over martial damage. I'd be inclined to think that this might, for example, leave a class like the Barbarian in the dust (though unlike Rogues, Barbarians often get additional sources of damage from their subclasses). If we were looking at only a vanilla Barbarian who just relies on their Rage bonus, Sneak Attack very clearly outpaces it.
Part of what I think balances out the several d6s of damage Rogues get with a Sneak Attack is the fact that they only get one attack (unless they're dual-wielding). But I also think this element of design has really held Rogues back - there's a certain "feel bad" element to the class even if the numbers did line up because of what I talked about in this post. In short, that more attacks "smooths out" your damage output by effectively creating a larger sample size.
My initial inclination was simply to lower the amount of damage a given Sneak Attack does, but then slap it on every attack. If we were adding, say, our PB (or the equivalent scaling with Rogue levels) number of d6s to each attack, this would ultimately wind up giving us slightly more damage by tier 4, but it would work a bit more like Rage damage, Radiant Strikes, or even Hex/Hunter's Mark.
If that were too much damage, though, I thought about making it a limited resource. Perhaps it's actually just as much damage, but the Rogue can only do it a certain number of times per Short Rest (feels right that it'd be a Short Rest resource, like a Monk's Focus Points).
But thinking about this in thematic terms, I actually think limiting Sneak Attack to once per turn feels right. Mechanically, this could represent a careful and patient sniper lining up a kill shot or a scrappy street-brawler feinting and ducking under their foe's arm while bringing the blade up into their ribs.
Now, making it a limited resource I do think actually kind of works on theme, but this also might be an unnecessary nerf to a class that already kind of struggles to keep up.
You know, the more I think about it, the more I'm inclined to the simpler solution. One real advantage here is that it also disrupts as little as possible. Cunning Strikes would still work just fine.
Let's take a step back, though:
Ultimately, what this proposal would do is primarily just add a second attack. Sneak Attack is not guaranteed, but I think for the most part, in most combats, you'll have some means to get it (most often just by having a friend near the target, though the Vex mastery might really change this. I just haven't played with a Rogue since 5.5 came out, except my own in a campaign where we weren't allowed to upgrade). This just simplifies things, and I actually think helps the Rogue with brief solo fights, which the class feels kind of inclined toward.
I think we'd need to consider some other breakpoints, though. Given that I haven't been doing a whole bunch of "reasonable fights at level 5" calculations, I don't know how useful it would be to actually do that for this version of the Rogue.
But my main inclination is that by giving them Extra Attack, you'd put Rogues on the same basic chassis as all the other martial classes, and then you could get a broad sense of whether Sneak Attack measures up to the effects of Rangers' and Paladins' spellcasting, Barbarians' Rage and subclass damage bonuses, and such, and as long as those seemed to be in the same ballpark, you'd probably be good.