I'm going to interpret your question in the language of Gowers's "two cultures" essay as follows:

> How does one get good at theory-building?

The process of developing a good theory can seem deceptively simple. One takes some definitions, perhaps by generalizing some known definitions, and deduces simple consequences of them. In comparison with the work required to solve a hard problem, this seems easy---perhaps too easy. The catch, of course, is the one you raised: there is a significant risk of spending a lot of time studying something that ultimately has very little mathematical value. Of course there is also the risk of wasted effort when trying to solve a specific problem, but in that case, it's at least clear what you were trying to accomplish. In the case of theory-building, the signposts are less clear; maybe you succeeded in proving some things, so your efforts weren't entirely fruitless, but at the same time, how do you know that you actually got somewhere when there was no clear endpoint?

The number one principle that I keep in mind when trying to build a theory is this:

> Relentlessly pursue the goal of understanding what's really going on.

I'm reminded of a wonderful sentence that Loring Tu wrote in his May 2006 Notices article on "The Life and Works of Raoul Bott." Tu wrote, "I. M. Singer remarked that in their younger days, whenever they had a mathematical discussion, the most common phrase Bott uttered was “I don't understand,” and that a few months later Bott would emerge with a beautiful paper on precisely the subject he had repeatedly not understood." Von Neumann reportedly said that in mathematics, you don't understand things; you just get used to them. This can be valuable advice to a young mathematician who hasn't yet grasped that the reason we're doing research is precisely that we don't really understand what we're doing. However, the key to theory-building is to insist on thorough understanding, especially of things that are widely regarded as being already understood. Often, such subjects are not really as well understood as others would have you believe. If you start asking probing questions---why are things defined this way and not that way? why doesn't this argument actually prove something more (or maybe it does?)?---you will find surprisingly often that what seems like a very basic question has not really been addressed before.

You asked:

> How do you decide whether a generalisation (that you find natural) of an established algebraic concept is worth studying? How convincing does the heuristic "well, X naturally generalises Y and we all know how useful Y is" sound to you?

My reply is that the generalization is worth studying if it helps you understand the original concept better. Perhaps the generalization was obtained by weakening an axiom, and you can now see more clearly that certain theorems hold more generally while others don't, so you get some insight into which specific hypotheses of your original object are needed for which conclusions. The heuristic as you've stated it, on the other hand, doesn't sound too convincing to me. I see too much risk of wandering off into a fruitless direction if you're not firmly grounded in trying to understand your original object better.

Keeping firmly in mind that your goal is a thorough understanding of some particular subject is also important because your efforts will, at least initially, not be greeted with enthusiasm by others. You will appear to be a complete idiot who doesn't understand even very basic things that other people think are obvious. Even when you start getting some fresh insights, they will seem trivial to others, who will claim that they "already knew that" (which they probably did, implicitly if not explicitly). Constantly adjusting definitions also appears to others to be an unproductive use of time. Even if you get to the point where your approach leads to a new and wonderfully clear presentation of the subject, and raises important new questions that nobody thought to ask before, you may not get credit for original thinking. Thus it is important that your internal compass is pointed firmly in the right direction. To repeat: ask yourself, am I driving towards an understanding of what's really going on in this important piece of mathematics? If so, keep at it. If not, then you've lost the thread somewhere along the way.

https://mathoverflow.net/a/71769/148161

> Long ago, when shepherds wanted to see if two herds of sheep were isomorphic, they would look for an explicit isomorphism. In other words, they would line up both herds and try to match each sheep in one herd with a sheep in the other. But one day, along came a shepherd who invented decategorification. She realized one could take each herd and "count" it, setting up an isomorphism between it and some set of "numbers", which were nonsense words like "one, two, three,..." specially designed for this purpose. By comparing the resulting numbers, she could show that two herds were isomorphic without explicitly establishing an isomorphism! In short, by decategorifying the category of finite sets, the set of natural numbers was invented.

Today with category theory go back to these roots.

https://mathoverflow.net/q/282854/148161

https://mathoverflow.net/q/953/148161 (see Scholze's answer)

https://annas-archive.org/

> Последний писк моды

> Недавно появился, делает поиск по всем базам (включая sci-hub), несколько раз находил там то, чего не было на libgen

(c) Павлов

Indeed, the five stages present a process of growth which end with the liberation from a difficult period in one's life, such as death, classical mathematics, drugs, or commutative groups. (с)

https://mathoverflow.net/a/25385/148161

You can discover the existence of rationally hyperbolic spaces by yourself. Try to do a rational homotopy calculation of a manifold that it not a homogeneous space, say hypersurfaces in CP3, where you know everything about the real cohomology ring. Moreover, these space are simply-connected and formal; so it looks like an easy exercise. After you filled several pages with generators and differentials, you begin to realize that the computation will not close off as elegantly as in the case of C^Pn or Sn. As a grad student, I spent a long night doing that -:))

https://mathoverflow.net/questions/44094/finiteness-of-higher-homotopy-groups-of-finite-complexes#comment105219_44101

Тимур, это для тебя

> As many of you know, category theorists are currently doing, among other things, a great job in advertising their modern developments. And I must say, this works for me - in particular, I find myself spending more and more time in nLab and the n-category cafe, fully understanding probably tiny bits of what they say there, and feeling that I really should get a grasp of the great picture at some point. My problem is, as for many students from the "analysis camp", almost total lack of background, except maybe some basic algebraic topology and even more basic algebraic geometry.

https://math.stackexchange.com/q/584013/656606