Gödel's incompleteness theorems align with the idea of finite trying to perfectly define infinite. A finite creature (human) trying to count to infinity is impossible, because no matter how high we count, we're no closer to infinity than when we started. This is similar to a finite creature (human) trying to come up with a finite system to perfectly define everything within an infinite system. It's impossible. Even though we can learn more and add to the finite system to better define the infinite, it will still be forever incomplete, because there will always be some things we won't know, or can't prove/disprove. Gödel's incompleteness theorems perfectly describe our position within the universe.

As an aside, it's interesting you bring this up, because I just recently was discussing it with someone, who was trying to use Gödel's incompleteness theorems to disprove something I said, which didn't work out for them, because they misunderstood what I was saying, and what Gödel's incompleteness theorems meant.