I suspect, really, that it's not so much that you're catching the same fish twice, as it is that each kind of catch comes in cycles.
Just pay a little attention to the patterns of your catches, and you'll see what I mean. It may not be so obvious while targetting something like moat carp, which has an excellent bite rate, and exists mostly in areas with a not very diverse pool of possible catches, but pretty much anything else...
Say you're targetting silver shark in Sauromugue Champaign with meatballs, for your daily guild points. Anyone who's ever done this knows that the bite rate is pretty bad, but then again, not exactly random. What usually happens is that you'll throw in about eight or nine lines with a "you didn't catch anything message," but then you'll catch not one, but two or three in a row. This is true of anything...black sole, bhefel marlin, you name it: fish come in teams. You rarely only get one of a certain kind, unless it's something really rare, like gold carp. You'll get two or three of the same type before the table cycles through to something else.
You can see this for yourself with other fishers, too. I know most people turn on the "other's catches and synths" filter, but if the curiosity ever hits you, leave it off in popular fishing spots, and pay particular attention to people targetting the same things you are. On the ferry with slices of bluetail, for example, you'll often notice that every time YOU hook a marlin, all of your neighbors with slices of bluetail will hook one, too. Every time you hook a silver shark, your neighbors will too. And, yes, every time you hook ryugu titan, even if your neighbors can't reel it in, you can plainly see from the way their rods behave that they've got one on the line too.
This goes for rusty items, as well. Stand next to somebody fishing moat carp in Windurst. Fish next to them for a while. When they get a gold carp, you will too. When they get three rusty items in a row, you probably will too.
It's simply cycling...I don't think that the game was actually programmed to think of it as "the same fish" when you let it go and the same kind comes back. Rather, I believe that there's a "quota," a certain number of a certain type of fish you can catch in a time period, and once that quota has been reached, it'll cycle to whatever has been generated next on the table. If the game decides that everyone has 45 seconds to catch one gold carp, then you'll keep hooking them until the time elapses or you get that one carp. If it decides everyone gets two minutes to catch two bhefel marlin...well, you get my point.
Of course, this does mean that there will be some slight variance if someone manages to fill the quota before the next time period approaches. I suspect that the table looks something like this:
[Period #1] [Period #2]
A[ 2:30 ] [ 1:10 ]
B[2 slvrshrk] [ rygttan ]
C[1 marlin ] [ cnclmry ]
D[ nothing ] [ monster ]
Where the columns represent the time period cycled through, the first row represents the time given for that period, and the following rows represent what's possible to catch in that time period. The items in the top rows will be hooked first, and will only go to the following rows if the quota has been reached. Once the time period has been elapsed, the next column moves up. Note that the last rows would be nigh impossible to approach before the time period elapsed without hacking.
So, in this example, you'd be given two minutes and thirty seconds to catch two silver shark, and you'd notice everyone else around you hooking silver shark as you did. This would probably take up the full time period for most people (as it was probably designed to do), but for the particularly high-level, fast, or just lucky fishermen, they might have filled the quota with time to spare...in which case the next row for that period would cycle up for them, and the next fish (in this case, a marlin) would bite. They'd likely be busy with that well into the next period, but would eventually synch with everyone again by hooking a titan as they cast their line.
Of course, people may point out that it's entirely possible to "skip" certain catches fairly quickly, ala the old "oh crapz this is a monster canc3lz!!!1". Once you hit escape, your next bite is completely different. I suspect that this actually means one of a number things:
1) Things like monsters, or particularly rare fish, can sometimes have extroardinarily short "time frames" in which to catch them...say, something in the range of five seconds. If you don't cast your line within that time period, you don't catch it, and you move on.
2) While the table may show one thing is up for you to hook, there may always be a random, small chance that you'll hook a monster or an item instead. I suspect that this is actually the case, as it backs up the patterns I've observed for monster catches while watching other fishermen on the ferry. Four of us might be fishing with slices of bluetail, three of us caught a marlin, but unlucky fisherman #4 got a monster.
3) It could mean something else entirely, or I'm completely wrong.
This all may seem involved, but as near as I can tell from my observations while fishing, this theory best explains everything that I can see. This is how I would do it myself, if were trying to emulate how FFXI fishing catches seem to be chosen.
If I'm right, you're not hooking the same gold carp twice. Instead, you've been given maybe a minute to catch one gold carp, and you'll keep hooking a gold carp until you catch it or the time runs out. Does that make sense?
