feat: Removed redundant User-agent which may cause conflict among cra…

…wlers
implerhq · Nov 20, 2024 · 8b94046 · 8b94046
1 parent 3ac7ee7
commit 8b94046
Showing 1 changed file with 2 additions and 5 deletions.
diff --git a/apps/web/public/robots.txt b/apps/web/public/robots.txt
@@ -1,7 +1,4 @@
-# https://www.robotstxt.org/robotstxt.html
-User-agent: *
+User-agent: *    
 Disallow:
-
-User-agent: *
 Allow: /auth/signin
-Allow: /auth/signup
+Allow: /auth/signup