Can LLMs Effectively Leverage Graph Structural Information: When and Why

LLMs,