I think that only holds true on the assumption that you follow main lines after 1 e4.  After 1 e4 e5 2 Nf3 Nc6, there's the Scotch, Goring, Evans, Two Knights and various central gambits.  After 2...Nf6 there's the Cochrane Gambit.  White can also deviate at move 2 with 2 d4 or the King's Gambit.
The Fort Knox can be side-stepped with the Tarrasch, Advance, or 3 Nc3 dxe4 4 f3.

I will concede, though, that objectively those gambits are easier to equalise against than most of the sharp gambits that can arise from 1 d4 d5 2 c4.  That said, it's much harder to find sharp tactical lines for White after 1 d4 Nf6 2 c4 e6 that are likely to provide an edge with accurate play.

Playing gambits and attacking does require some positional understanding to yield much success above a certain level.   I completely agree with the last part of MNb's post above.  In addition most gambits can be declined safely, so you need to know how to generate winning chances from relatively balanced tactical positions.

I think of playing BDG and nothing else as a very narrow way of playing- of course, each to their own, if that's what you enjoy then that's fine, but it does mean you get similar positions in pretty much every game.  Therefore your understanding of chess ends up stunted and you're more likely to come unstuck when opponents "duck out" with a line like 1 d4 d5 2 e4 e6 3 Nc3 (or 2 Nc3 e6 3 e4) Bb4, putting a stop to White's e4/f3-gambit ideas.

Actually, when I first started chess I wasn't that much of a gambiteer, I used to play the standard openings juniors are taught (Italian Game, Fried Liver, Open Sicilian) and dabbled in the Queen's Gambit and Torre Attack, and played seriously in tournaments.  It was when I rediscovered my chess-playing hobby in 2004 that I decided to just play for pleasure and started experimenting with various gambit lines, resulting in me adding a large assortment of gambits to my repertoire.

However I'm primarily an 1 e4-player so I use the BDG only as an occasional weapon, usually via transposition from certain 1 e4-openings (although I do occasionally wheel out 1 d4, 2 Nc3 and 3 e4, when in practice many of my opponents transpose back to a French or Pirc). 

I agree about the distinction between serious and fun chess.  Of course, different people find different aspects of chess 'fun', but as in most competitive activities, the majority tend to find more 'fun' in risky and aggressive play than in technical grinding play.

If BDG-fans want a sharp game, then why don't they just play 1.e4?

I'm grateful, MNb, for your very thoughtful and candid comments.  My question was inspired by my preparing for my first otb tournament in more than a decade and trying to think about what opening preparations were necessary based on a variety of possibilities from my opponent.  That, combined with a number of the brasher comments here, left me wondering if transposing from a BDG to a French or Caro would result in my opponent throwing the chessboard at me and calling me a coward or--more to the point--to what extent I could expect him/her to be well-prepared for my "cowardice."

This is a sincere question and not one aimed at provoking die-hard BD gambiteers, but all this talk about machismo and cowardice associated with playing or declining gambits has me wondering whether players bent on gambits actually know any theory beyond their pet variations and what positional understanding they might have.

Is declining the BDG cowardice or taking the game along paths that the gambiteer does not know so well?  In the past, I've played the French or Caro-Kann (two openings I played against 1.e4) against the BDG, and I won very easily.  Do BDG players consider these more "traditional" alternatives and prepare for them accordingly?

Moreover, aside from the precarious imbalances they seek to acquire over the board, do BDG players understand the qualities of space, central control, piece activity, etc., or do they prey primarily upon the chaos on the board?