Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redesign notice handling and introduce NoticeProcessorPipeline #1369

Merged
merged 54 commits into from
Sep 1, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
e0e5b6f
Add contracts to `cast`, `safeCast` and `castOrNull`; Add `uncheckedC…
Him188 Jun 25, 2021
7a34267
Add `out` variance to generic type of `ContactList`
Him188 Jun 26, 2021
46b0fee
Use `NormalMemberImpl` for properties of `GroupImpl`
Him188 Jun 26, 2021
7fc58e3
Use internal types in implementations of properties in `Bot`
Him188 Jun 26, 2021
980e2c8
Add `TypeSafeMap`
Him188 Jun 26, 2021
1cd421b
Redesign notice handling and introduce `NoticeProcessorPipeline`
Him188 Jun 26, 2021
35b835c
Fix incorrect coroutine scope created for contacts in `IMirai.newFrie…
Him188 Jun 27, 2021
c170919
Redesign notice handling and introduce `NoticeProcessorPipeline` part 2
Him188 Jun 27, 2021
ea2bb3a
Add `UnconsumedNoticesAlerter` and cleanup
Him188 Jun 27, 2021
22cc78e
Improve performance
Him188 Jun 27, 2021
0160be3
Remove `Group.newAnonymousMember`
Him188 Jun 28, 2021
8c171ad
Fix `ByteArray.loadAs`
Him188 Jun 28, 2021
0859b07
Redesign notice handling and introduce `NoticeProcessorPipeline` part 3:
Him188 Jun 28, 2021
eacb15c
Extract utils into multiple files
Him188 Jun 29, 2021
479f2b6
apiDump
Him188 Jun 29, 2021
c6de8c6
Delete empty ProtoBuf.kt
Him188 Jul 3, 2021
492257f
Move `bkn` from `QQAndroidBot` to `WLoginSigInfo`
Him188 Aug 13, 2021
6aef4da
Remove outdated todo in `FriendNoticeProcessor`
Him188 Aug 13, 2021
ab5f2e5
Add `MutableProcessResult`; remove locks
Him188 Aug 13, 2021
af2e71b
Migrate more transformers to `NoticeProcessorPipeline`
Him188 Aug 13, 2021
bee764b
Migrate more transformers to `NoticeProcessorPipeline`
Him188 Aug 14, 2021
a831eb7
Ensure all items in `MultiPacket` are visited
Him188 Aug 14, 2021
82118cb
Remove `.debug` in `mirai.network.debug.notice.pipeline.log.full`
Him188 Aug 16, 2021
de48274
Avoid boxing in `TypeSafeMap` and support serialization
Him188 Aug 16, 2021
cf3de70
Add simple constructor for `TypeSafeMap`
Him188 Aug 16, 2021
d01407d
Add `TraceLoggingNoticeProcessor`
Him188 Aug 16, 2021
d511611
Make `UnconsumedNoticesAlerter` disabled by default
Him188 Aug 16, 2021
4adaa13
Fix `EventBroadcasterPacketHandler.impl`
Him188 Aug 16, 2021
8852f50
Add `AbstractPipelineContext`
Him188 Aug 16, 2021
e1837bd
Add `RecordingNoticeProcessor`
Him188 Aug 16, 2021
54c9c73
Add gradle configuration "use.maven.local"
Him188 Aug 16, 2021
f892b65
Fix `ParseErrorPacket` not logged
Him188 Aug 16, 2021
665467b
Add `toString` for TypeSafeMap implementations
Him188 Aug 16, 2021
c50ac7d
Make `TypeSafeMap()` returns `TypeSafeMap.EMPTY`
Him188 Aug 25, 2021
e433d6b
Add util `readResource`
Him188 Aug 25, 2021
84ad2da
Add util `CollectionDiff`
Him188 Aug 25, 2021
e79a907
Add util `Array<Byte>.toUHexString`
Him188 Aug 25, 2021
a1c0722
Add ConstructorCallCodegen
Him188 Aug 25, 2021
b7a54d9
Redesign packet recording
Him188 Aug 25, 2021
75d3883
Fix group transfer and retrieve, add tests for notice events:
Him188 Aug 25, 2021
70f2b04
Use yamlkt 0.10.2
Him188 Aug 25, 2021
b0911f7
Fix `MemberJoinEvent` processing and add `MemberJoinTest`
Him188 Aug 27, 2021
0c8c576
Add test for message events
Him188 Aug 27, 2021
d4a02d6
Fix `RecordingNoticeProcessorTest`
Him188 Aug 27, 2021
324e15a
Get group by uin in `OnlineMessageSourceFromTempImpl`, fix the proble…
Him188 Aug 27, 2021
ff608b6
Add comments for kick event
Him188 Aug 27, 2021
003bbd4
Handle possible `MemberLeaveEvent.Kick` for `NewContact.SystemMsgNewG…
Him188 Aug 27, 2021
fb6335d
Ignore `NewContact.SystemMsgNewGroup, subType=0, groupMsgType=8`, fix…
Him188 Aug 27, 2021
5283843
Ignore `NewContact.SystemMsgNewGroup, subType=5, groupMsgType=16`, fi…
Him188 Aug 27, 2021
c7ff883
Support nested structures in desensitizer
Him188 Aug 27, 2021
7565d03
Remove more verbose default values in ConstructorCallCodegen
Him188 Aug 27, 2021
12228d7
Add `BotInvitedJoinTest`
Him188 Aug 27, 2021
4bb751d
Use `getGroupByUinOrCodeOrFail` in OnlineMessageSourceFromTempImpl fo…
Him188 Aug 27, 2021
2cc101e
Fix FriendAvatarChangedEvent
Him188 Aug 30, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Ensure all items in MultiPacket are visited
  • Loading branch information
Him188 committed Aug 25, 2021
commit a831eb71f6eb36ea295ff4214a29557221ed2aaa
36 changes: 31 additions & 5 deletions mirai-core/src/commonMain/kotlin/network/Packet.kt
Original file line number Diff line number Diff line change
Expand Up @@ -33,22 +33,48 @@ interface Packet {
/**
* PacketFactory 可以一次解析多个包出来. 它们将会被分别广播.
*/
internal interface MultiPacket : Packet, Collection<Packet>
internal interface MultiPacket : Packet {
/**
* `true` if this packet has some useful meaning, otherwise it will be considered just as a wrapper of its children.
*/
val isMeaningful: Boolean

/**
* if item is [MultiPacket], its children will be ignored.
*/
fun children(): Iterator<Packet>
}

internal fun Collection<Packet>.toPacket(): Packet {
return when (this.size) {
1 -> this.single()
else -> MultiPacketImpl(this)
else -> MultiPacket(this)
}
}

internal fun MultiPacket(delegate: Collection<Packet>): MultiPacket = MultiPacketImpl(delegate)
internal fun MultiPacket(delegate: Collection<Packet>): MultiPacket {
return MultiPacketImpl(delegate)
}

internal fun MultiPacket(delegate: Packet): MultiPacket =
if (delegate is MultiPacket) delegate else MultiPacket(listOf(delegate))

internal open class MultiPacketImpl(

private class MultiPacketImpl(
val delegate: Collection<Packet>,
) : MultiPacket, Collection<Packet> by delegate {
) : MultiPacket {
override val isMeaningful: Boolean get() = false

override fun children(): Iterator<Packet> {
return sequence {
for (packet in delegate) {
yield(packet)
if (packet is MultiPacket) {
yieldAll(packet.children())
}
}
}.iterator()
}

override fun toString(): String = delegate.joinToString(
separator = "\n",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,8 @@ internal class EventBroadcasterPacketHandler(
private fun impl(packet: Packet?) {
if (packet == null) return
if (packet is MultiPacket) {
for (p in packet) {
if (packet.isMeaningful) impl(packet)
for (p in packet.children()) {
impl(p)
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,8 @@ internal class PacketLoggingStrategyImpl(
}

if (packet is MultiPacket) {
for (d in packet) {
if (packet.isMeaningful) logReceivedImpl(packet, incomingPacket, logger)
for (d in packet.children()) {
logReceivedImpl(d, incomingPacket, logger)
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -81,13 +81,17 @@ internal object MessageSvcPbGetMsg : OutgoingPacketFactory<MessageSvcPbGetMsg.Re
*/
open class Response(
internal val syncFlagFromServer: MsgSvc.SyncFlag,
delegate: List<Packet>,
private val delegate: List<Packet>,
val syncCookie: ByteArray?, override val bot: Bot,
) : AbstractEvent(),
MultiPacket,
Collection<Packet> by delegate,
Packet.NoEventLog,
BotEvent {
override val isMeaningful: Boolean get() = true

override fun children(): Iterator<Packet> {
return delegate.iterator()
}

override fun toString(): String =
"MessageSvcPbGetMsg.Response(flag=$syncFlagFromServer)"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,25 +11,16 @@ package net.mamoe.mirai.internal.network.protocol.packet.chat.receive

import kotlinx.io.core.ByteReadPacket
import net.mamoe.mirai.internal.QQAndroidBot
import net.mamoe.mirai.internal.contact.GroupImpl
import net.mamoe.mirai.internal.network.MultiPacket
import net.mamoe.mirai.internal.network.Packet
import net.mamoe.mirai.internal.network.components.NoticeProcessorPipeline.Companion.processPacketThroughPipeline
import net.mamoe.mirai.internal.network.protocol.data.jce.MsgInfo
import net.mamoe.mirai.internal.network.protocol.data.jce.MsgType0x210
import net.mamoe.mirai.internal.network.protocol.data.jce.OnlinePushPack
import net.mamoe.mirai.internal.network.protocol.data.proto.Submsgtype0x122
import net.mamoe.mirai.internal.network.protocol.packet.IncomingPacketFactory
import net.mamoe.mirai.internal.network.protocol.packet.OutgoingPacket
import net.mamoe.mirai.internal.network.protocol.packet.buildResponseUniPacket
import net.mamoe.mirai.internal.utils._miraiContentToString
import net.mamoe.mirai.internal.utils.io.serialization.loadAs
import net.mamoe.mirai.internal.utils.io.serialization.readUniPacket
import net.mamoe.mirai.internal.utils.io.serialization.writeJceRequestPacket
import net.mamoe.mirai.utils.debug


//0C 01 B1 89 BE 09 5E 3D 72 A6 00 01 73 68 FC 06 00 00 00 3C
internal object OnlinePushReqPush : IncomingPacketFactory<OnlinePushReqPush.ReqPushDecoded>(
"OnlinePush.ReqPush",
"OnlinePush.RespPush",
Expand Down Expand Up @@ -65,165 +56,4 @@ internal object OnlinePushReqPush : IncomingPacketFactory<OnlinePushReqPush.ReqP
)
}
}
}


internal interface Lambda732 {
operator fun invoke(pk: ByteReadPacket, group: GroupImpl, bot: QQAndroidBot): Sequence<Packet>
}

internal inline fun lambda732(crossinline block: ByteReadPacket.(GroupImpl, QQAndroidBot) -> Sequence<Packet>): Lambda732 {
return object : Lambda732 {
override fun invoke(pk: ByteReadPacket, group: GroupImpl, bot: QQAndroidBot): Sequence<Packet> {
return block(pk, group, bot)
}
}
}

internal object Transformers732 : Map<Int, Lambda732> by mapOf(
// mute
0x0c to lambda732 { group: GroupImpl, bot: QQAndroidBot ->
TODO("removed")
},

// anonymous
0x0e to lambda732 { group: GroupImpl, _: QQAndroidBot ->
TODO("removed")
},

//系统提示
0x14 to lambda732 { group: GroupImpl, bot: QQAndroidBot ->
TODO("removed")
},
// 传字符串信息
0x10 to lambda732 { group: GroupImpl, bot: QQAndroidBot ->
TODO("removed")
/*
val dataBytes = readBytes(26)

when (dataBytes[0].toInt() and 0xFF) {
59 -> { // TODO 应该在 Transformers528 处理
val size = readByte().toInt() // orthodox, don't `readUByte`
if (size < 0) {
// java.lang.IllegalStateException: negative array size: -100, remaining bytes=B0 E6 99 90 D8 E8 02 98 06 01
// java.lang.IllegalStateException: negative array size: -121, remaining bytes=03 10 D9 F7 A2 93 0D 18 E0 DB E8 CA 0B 32 22 61 34 64 31 34 64 61 64 65 65 38 32 32 34 62 64 32 35 34 65 63 37 62 62 30 33 30 66 61 36 66 61 6D 6A 38 0E 48 00 58 01 70 C8 E8 9B 07 7A AD 02 3C 7B 22 69 63 6F 6E 22 3A 22 71 71 77 61 6C 6C 65 74 5F 63 75 73 74 6F 6D 5F 74 69 70 73 5F 69 64 69 6F 6D 5F 69 63 6F 6E 2E 70 6E 67 22 2C 22 61 6C 74 22 3A 22 22 7D 3E 3C 7B 22 63 6D 64 22 3A 31 2C 22 64 61 74 61 22 3A 22 6C 69 73 74 69 64 3D 31 30 30 30 30 34 35 32 30 31 32 30 30 34 30 38 31 32 30 30 31 30 39 36 31 32 33 31 34 35 30 30 26 67 72 6F 75 70 74 79 70 65 3D 31 22 2C 22 74 65 78 74 43 6F 6C 6F 72 22 3A 22 30 78 38 37 38 42 39 39 22 2C 22 74 65 78 74 22 3A 22 E6 8E A5 E9 BE 99 E7 BA A2 E5 8C 85 E4 B8 8B E4 B8 80 E4 B8 AA E6 8B BC E9 9F B3 EF BC 9A 22 7D 3E 3C 7B 22 63 6D 64 22 3A 31 2C 22 64 61 74 61 22 3A 22 6C 69 73 74 69 64 3D 31 30 30 30 30 34 35 32 30 31 32 30 30 34 30 38 31 32 30 30 31 30 39 36 31 32 33 31 34 35 30 30 26 67 72 6F 75 70 74 79 70 65 3D 31 22 2C 22 74 65 78 74 43 6F 6C 6F 72 22 3A 22 30 78 45 36 32 35 35 35 22 2C 22 74 65 78 74 22 3A 22 64 69 6E 67 22 7D 3E 82 01 0C E8 80 81 E5 83 A7 E5 85 A5 E5 AE 9A 88 01 03 92 01 04 64 69 6E 67 A0 01 00
// negative array size: -40, remaining bytes=D6 94 C3 8C D8 E8 02 98 06 01
error("negative array size: $size, remaining bytes=${readBytes().toUHexString()}")
}

// println(dataBytes.toUHexString())
//println(message + ":" + dataBytes.toUHexString())

val new = when (val message = readString(size)) {
"管理员已关闭群聊坦白说" -> false
"管理员已开启群聊坦白说" -> true
else -> {
bot.network.logger.debug { "Unknown server messages $message" }
return@lambda732 emptySequence()
}
}

// @Suppress("DEPRECATION")
// if (group.settings.isConfessTalkEnabled == new) {
// return@lambda732 emptySequence()
// }

return@lambda732 sequenceOf(
GroupAllowConfessTalkEvent(
new,
false,
group,
false
)
)
}

0x2D -> {
// 修改群名. 在 Transformers528 0x27L 处理
return@lambda732 emptySequence()
}
else -> {
/*
bot.network.logger.debug("unknown Transformer732 0xunknown type: ${dataBytes[0].toString(16)
.uppercase()}")
bot.network.logger.debug("unknown Transformer732 0xdata= ${readBytes().toUHexString()}")
*/
return@lambda732 emptySequence()

/*
if (group.name == message) {
return@lambda732 emptySequence()
}

return@lambda732 sequenceOf(
GroupNameChangeEvent(
group.name.also { group._name = message },
message, group, false
)
)*/
}
}*/
},

// recall
0x11 to lambda732 { group: GroupImpl, bot: QQAndroidBot ->
TODO("removed")
},
)

internal interface Lambda528 {
suspend operator fun invoke(msg: MsgType0x210, bot: QQAndroidBot, msgInfo: MsgInfo): Sequence<Packet>
}

internal inline fun lambda528(crossinline block: suspend MsgType0x210.(QQAndroidBot) -> Sequence<Packet>): Lambda528 {
return object : Lambda528 {
override suspend fun invoke(msg: MsgType0x210, bot: QQAndroidBot, msgInfo: MsgInfo): Sequence<Packet> {
return block(msg, bot)
}

}
}

internal inline fun lambda528(crossinline block: suspend MsgType0x210.(QQAndroidBot, MsgInfo) -> Sequence<Packet>): Lambda528 {
return object : Lambda528 {
override suspend fun invoke(msg: MsgType0x210, bot: QQAndroidBot, msgInfo: MsgInfo): Sequence<Packet> {
return block(msg, bot, msgInfo)
}

}
}

// uSubMsgType to vProtobuf
// 138 or 139: top_package/akln.java:1568
// 66: top_package/nhz.java:269
/**
* @see MsgType0x210
*/
@OptIn(ExperimentalStdlibApi::class)
internal object Transformers528 : Map<Long, Lambda528> by mapOf(

0x8AL to lambda528 { bot ->
TODO("removed")
},

//戳一戳信息等
0x122L to lambda528 { bot, msgInfo ->
val body = vProtobuf.loadAs(Submsgtype0x122.Submsgtype0x122.MsgBody.serializer())
when (body.templId) {
//戳一戳
1132L, 1133L, 1134L, 1135L, 1136L, 10043L -> {
TODO("removed")
}
else -> {
bot.logger.debug {
"Unknown Transformers528 0x122L template\ntemplId=${body.templId}\nPermList=${body.msgTemplParam._miraiContentToString()}"
}
return@lambda528 emptySequence()
}
}
},
// 群相关, ModFriendRemark, DelFriend, ModGroupProfile
0x27L to lambda528 { bot ->
TODO("removed")
},
)
}