Refactor auto-sync updater
This refactors the auto-sync updater scripts, adds multiple tests and some other smaller things:
- Converts the updater in a proper Python package.
- Renaming was done to fit this new package structure.
- Format code with usort and black and enforce it with the CI.
- Add license information to auto-sync scripts.
- Update tree-sitter-cpp to v20.0.5
- Fix py-tree-sitter version to `< 0.22.0` due to https://github.com/tree-sitter/tree-sitter-cpp/issues/250
- Allow file/dir creation of non existing paths.
- Add CI tests for Patch, inc gen, translation and diff persistence testing.
- Implement editing of diffs with an editor.
- Fix: Add Namespace id also to anonymous enumeration members.
* Update sysop inc file
* Fix missing braces warning
* Handle new system operands
* Fix build errors by renaming.
* Fix segfault
* Fix segfault
* Add custom MCOperand valiadtors
* Add AArch64 case for getFeatureBits
* Fix infinite loop
* Fix braces warning.
* Implement loopuo by name for sys operands
* Fix incorrect translation which remove else if statements.
* Fix several segfaults
* Rename GetRegFromClass patch
* Fix segfaults and asserts
* Fix segfault
* Move MRI setting to Mapping
* Remove unused code
* Add add_op_X functinos for AArch64.
* Add fill detail functins
* Handle RegWithShiftExtend operands
* Handle TypedVectorList operands.
* Handle ComplexRoatation operands
* Handle MemExtend operands
* Handle ImmRangeScale operands
* Handle ExactFPImm operands
* Handle GPRSeqPairsClass operands
* Handle Imm8OptLsl operands
* Handle ImmScale operands
* Handle LogicalImm operands
* Handle Matrix operands
* Handle SME Matrix tiles and vectors.
* Handle normal operands.
* Fix segfault.
* Handle PostInc operands.
* Reorder VecLayout enum to have no duplicate enum value.
* Handle PredicateAsCounter operands
* Handle ZPRasFPR operands
* Handle VectorIndex operands
* Handle UImm12Offset operands.
* Move reg suffix to enum val to single function.
* Handle SVERegOp operands
* Handle SVELogicalImm operands
* Handle SImm operand
* Handle PrefetchOp operands
* Handle Imm and ImmHex operands
* Handle GPR64as32 and GPR64x8 operands
* Add missing break
* Handle FPImm operand
* Handle ExtendedRegister opreand
* Handle CondCode operands
* Handle BTIHintOp operands
* Handle BarrierOption operands
* Handle BarrierXSOption
* Add not implemeted case again
* Handle ArithExtend operands
* Handle AdrpLabel and AlignedLabel operands
* Handle AMNoIndex operands
* Handle AddSubImm operands
* Handle MSRSystemRegisters and MRSSystemRegister operands
* Handle PSBHntOp and RPRFMOperand operands
* Remove unused variables
* Handle InverseCondCode operands
* Handle ImplicityTypedVectorList operands
* Handle ShiftedRegister operands
* Handle Shifter operands
* Handle SIMDType10Operand operands
* Handle SVCROp operands
* Handle SVEPattern operands
* Handle SVEVecLenSpecifier operands
* Handle SysCROperands
* Handle SysXzrPair operands
* Handle PState operands
* Handle VRegOperands
* Primt SME oeprands.
* Fix cs_operand.h include
* Rename arm64 -> aarch64 in python bindings.
* Add Python bindings for SH
* Fix ARM Python bindings (#2127)
* Restructure auto-sync update scripts.
* Move Helper functions to Updater dir
* Move requirements.txt
* Add basic ASUpdater.py
* Run black.
* Add inc file generater to updater
* Add option to select certain inc files fore generation.
* Enable clean build and implement patcher for inc files.
* Format config
* Patch main header files after inc generation.
* Implement clang-format function (unused yet, because it takes forever.)
* Copy generated inc files to arch dir
* Invert clean option (noramlly we need to clean the build dir.)
* Clearify arg doc
* Rename SystemRegister file for AArch64
* Centralize handling of path variables.
* Check if SystemOperands had to be generated before renaming on of its files.
* Replace class parameters by calling get_path
* Remove updater config which only contained paths.
* Add refactor option.
* Remove more path handling in the Configurator.
* Add translation step to updater.
* Fix includes after CppTranslator was moved into the Updater
* Remove updater config
* Fix several issue in the Configurator
* Fix file operations
* Remove addition argument from translator.
* Add Differ step to updater.
* Add path variable for arch_config
* Add diff step.
* Fix typo
* Introduce .clang-format path variable.
* Remove duplicate functions
* Add option to select update steps to execute.
* Check in write functions for write flag.
* Rename PatchMainHeader -> HeaderPatcher
* Move .gitignore
* Add README to vendor dir.
* Add all system operands to cstool output
* Update cstest with aarch64 changes
* Remove wb flag of aarch64 detail struct
* Set updates_flag after decoding
* Set writeback after decoding.
* Rename ARM64 -> AArch64
* Update printer and op mapping
* Exit normally
* Add AArch64 alias
* Fix some tmeplate function calls
* Fix flag check after rebase.
* Fix build by commentig unnused code.
* Add memory operand flag
* Handle memory operands printed via generic printOperand function.
* Handle UImm memory offsets
* Introduce MEM_REG and MEM_IMM op types
* Handle scaled memory immediates
* Check for op_count before checking for mem op at -1 index.
* Update memory operand flags.
* Pass imm/reg memory ops in set_imm/reg to set_mem.
* Add missing set_sme_operand call and fix assert.
* Remove CS_OP_MEM flag before entering switch.
* Preidcates are registers.
* Add shift info always to the previous operand
* Check for generic system regs
* Handle NumLanes = 0 LaneKind = q case
* Replace printImm call with normal print logic. Otherwise ops get added twice to detail.
* Handle FP operands in printOperand.
* Add access information to float operands.
* Rewrite SME matrix handling.
* Set correct SME layouts and allow for immediate range sme offsets.
* Handle cases of unknown system alias by setting their raw values
* Update cstool and header file with new SME offset handling
* Handle SME Tile lists.
* Fix build error in cstest
* Update MC tests for AArch64
* Handle TLBI operands and fix printing bug.
* Fix: Print signed value as signed.
* Add more system alias to detail.
* Remove duplicate hex prefix
* Set correct values for the register info
* Replace tabs with white spaces
* Move string append logic to own function.
* Set DecodeComplete = true before decoding (as originally in the LLVM code).
* Change type of feature argument, since only LLVM features are passed, not CS groups.
* Imitate lower_bound for the index table binary search.
* Remove trailing comments from test files.
* Print shift amount in decimal
* Save detail of shift alias instructions.
* Add extension details fot ext instruction alias
* Print LSB and width in decimal
* Fix LLVM bug. The feature check for V8_2a doesn't check if all features are enabled.
* Fix lower_bounds check.
For m == 0 we wrap around 0 of cause.
* Fix feature check. Add check for FeatureAll since it includes XS
* Operate on temporary MCInst when trying decoding.
* Add lower_bound behavior to IndexTypeStr binsearch.
* Fix MC tests which were incorrect because of missing FeatureAll check
* Add Alias handling for AArch64
* Update system operands with SYSIMM types and add additional sysop category.
* Add macros for meta programming (ARM64 <-> AArch64 selection).
* Fix union/struct confusion and add raw_value member to uninions.
* Allow to set Syntax and mode options for AArch64
* Fix build warning by using correct type
* Print shift value in decimal
* Add missing call to add_cs_detail.
* Update name map files with normalized names.
* Remove unused function
* Add check if detail should be filled.
* Fill detail for real instructions if only real detail is requested.
* Add always the extension.
* Make dir creation log message debug level
* Implement ADR immediate operand printer.
See: c3484b1fdc
* Check for flag registers beeing written and update flag.
* Move multiple CondCode helpers to aarch64.h because they are so freaking useful.
+ Print CC if it is EQ
* Fix incorrectly initialized CC and VectorLayout.
* Add LSL shift type for extensions.
* Fix case when shift amount is 0
* Fix post-index memory instructions.
* Pass raw immediate through getShiftValue to extract actual shift amount
* Setup AArch64 detail ops.
* Add flag for operands part of a list.
* Set vector indices for all relevant registers.
* Add missing call to add_cs_detail for postIncOperands
* Add ugly yet reliable way to determine post-index addressing mode
* Add support for old Capstone register alias.
* Remove leading space before some alias mnemonics.
* add AARCH64 to `cmake.sh`
* add HAS_AARCH64 to `cs.c`
* should probably just reference `cs_operand.h` in `aarch64.h`
* hint compiler at `AArch64_SYSREG` enum type for casting purposes
* update `Makefile` for AARCH64
leaves `CAPSTONE_HAS_ARM64` supported
* `testFeatureBits` platform function check
`testFeatureBits` should check if the platform function is visible first
* update tests to use AARCH64 convention
* hack: avoid enum casts for `MCInst` Values
Apple compiler really hates typecasting a enum, even if bounded from a unsigned. Lets set the raw_value directly
is a hack and needs proper review
* Check for present detail before accessing it.
* Add CS only groups
* Use general map ins_op type
* Fix build warning about str size computation.
* Disable warning about unitialized value for GCC 11.
Imm is initialized and the warning does not appear
in later versions.
* Use correct include guard for PPC
* Add missing requirements
* Update SystemOperand enums.
* Fix overlapping comparison warning
* Fix reachable assert where OpNum is not of type IMM
* Handle 0.0 operand for fcmp
* Fix incorrect variable passed.
* Fix for MacOS which doesn't know the warning and throws another one.
* Make getExtendEncoding static to fix build warning on MSVC.
* Fix build error: 'missing binary operator before token' by checking __GNUC__
* Add string search to add vector layout info.
* Add missing mem disponents of several ldr and str instructions.
* Add 0 immediates to several instructions.
* Rename v regs to q and d variant.
The cs_regname API can not pass the variant name of the register requested.
So we simply emit the default variant name.
* Fix incorrect enum value.
* Fix tests for system operands.
* Fix syntax issues in tests.
* Rename Arm64 -> AArch64 Python bindings.
* Fix Python bindings C structs.
* Fix generation of constants (ARMCC skipped because it starts with ARM)
* Update const files
* Remove -Wmaybe-uninitialized warning since it fails fuzz build
* Add missing comma
* Fix case
* Fix AArch64 Python bindings:
- Do not generate constants automatically (dscript is way too buggy).
- Update printing of details.
* Rename ARM64 -> AArch64 in test_corpus.py
* Rename test_arm64 -> test_aarch64
* Rename ARM-64 -> AArch64
* Fix diff CI test by disassembling AArch64 at former ARM64 place
* Fix several wrong types and remove unnecessary memebers from Python binding
* Fix: Same printing format of detail for cstool, test_ and test_*.py
* Fix: pass correct op index for mov alias with op[1] == reg wzr.
* Set prfm op manuall in case of unnown sysop. set_imm would add it to an memory operand wihtout base.
* Fix: If barrier ops are not set an assert is reached.
We fix it here by simply getting the immediate as the printing code does.
---------
Co-authored-by: Peace-Maker <peace-maker@wcfan.de>
Co-authored-by: Dayton <5340801+watbulb@users.noreply.github.com>
* Add venv to the gitignore so I can test Python code
* add test_iter.py which is nearly identical to test_lite.py
* add support for test_iter.py in check; also add missing test_lite.py
* Add support for the disasm_iter C function
* Added RISCV dir to contain the RISCV architecture engine code. Adding the TableGen files generated from llvm-tblgen. Add Disassembler.h
* Started working on RISCVDisassembler.c - RISCV_init(), RISCVDisassembler_getInstruction, and RISCV_getInstruction
* Added all functions to RISCVDisassembler.c and needed modifications to RISCVGenDisassemblerTables.inc. Add and modified RISCVGenSubtargetInfo.inc. Start creation of RISCVInstPrinter.h
* Finished RISCVGenAsmWriter.inc. Finished RISCVGenRegisterInfo.inc. Minor fixes to RISCVDisassembler.c. Working on RISCVInstPrinter
* Finished RISCVInstPrinter, RISCVMapping, RISCVBaseInfo, RISCVGenInstrInfo.inc, RISCVModule.c. Working on riscv.h
* Backport it from: 0db412ce3b
* All RISCV files added. Compiled correctly and initial test for ADD, ADDI, AND works properly.
* Add refactored cs.c for RISCV
* Testing all I instructions in test_riscv.c
* Modify the orignal backport for RISCVGenRegisterInfo.inc, capstone.h and test_iter to work w/ the current code strcuture
* Fix issue with RISCVGenRegisterInfo.inc - RISCVRegDesc[] (Excess elements in struct initializer). Added RISCV tests to test_iter.c
* fixed bug related to incorrect initialization of memory after malloc
* fix compile bug
* Fix compile errors.
* move riscv.h to include/capstone
* fix indentation issues
* fix coding style issues
* Fix indentation issues
* fix coding style
* Move variable declaration to the top of the block
* Fix coding indentation
* Move some stuff into RISCVMappingInsn.inc
* Fix code sytle
* remove cs_mode support for RISCV
* update asmwriter-inc to LLVM upstream
* update the .inc files to riscv upstream
* update riscv disassembler function for suport 16bit instructions
* update printer & tablegen inc files which have fixed arguments mismatch
* update headers and mapping source
* add riscv architecture specific test code
* fix all RISCV tons of compiler errors
* pass final tests
* add riscv tablegen patchs
* merge with upstream/next
* fix cstool missing riscv file
* fix root Makefile
* add new TableGen patchs for riscv
* fix cmakefile.txt of missing one riscv file
* fix declaration conflict
* fix incompatible declaration type
* change riscvc from arch to mode
* fix test_riscv warnning
* fix code style and add riscv part of test_basic
* add RISCV64 mode
* add suite for riscv
* crack fuzz test
* fix getfeaturebits test add riscvc
* fix test missing const qualifier warnning
* fix testcase type mismatch
* fix return value missing
* change getfeaturebits test
* add test cs files
* using a winder type contain the decode string
* fix a copy typo
* remove useless mode for riscv
* change cs file blank type
* add repo for update_riscv & fix cstool missing riscv mode
* fix typo
* add riscv for cstool useage
* add TableGen patch for riscv asmwriter
* clean ctags file
* remove black comment line
* fix fuzz related something
* fix missing RISCV string of fuzz
* update readme, etc..
* add riscv *.s.cs file
* add riscv *.s.cs file & clear ctags
* clear useless array declarations at capstone_test
* update to 5e4069f
* update readme change name more formal
* change position of riscv after bpf and modify copyright more uniform
* clear useless ctags file
* change blank with tab in riscv.h
* add riscv python bindings
* add riscv in __init__.py
* fix riscv define value for python binding
* fix test_riscv.py typo
* add missing riscvc in __init__.py of python bindings
* fix alias-insn printer bug, remove useless newline
* change inst print delimter from tab to bankspace for travis
* add riscv tablegen patch
* fix inst output more consistency
* add TableGen patch which fix inst output formal
* crack the effective address output for detail and change register print function
* fix not detail crash bug
* change item declaration position at cs_riscv
* update riscv.py
* change function name more meaningfull
* update python binding makefile
* fix register enum sequence according to riscvgenreginfo.inc
* test function name
* add enum s0/fp in riscv.h & update riscv_const.py
* add register name enum
* Basic changes of new arch - BPF
* Define some constants
* defined some API methods
* Able to print MISC instruction
* Follow Linux coding style
* Ability to show ALU insn names
* decode return
* Add suite/MC/BPF
* decode jump
* decode store
* decode load
* print instruction done
* try to implement BPF_reg_access
* Implements explicit accessed registers and fix some tiny bugs
* Fix unhandled ja case
* Added BPF_REG_OFF do fix wrong display in jump class
* Great I'm able to decode cBPF with eyes
* Fix: misunderstood the 16-byte instruction's imm
* Add ldxdw
* Add extended-all.cs
* Implements cstest/bpf_getdetail.c
* Fix memory leak
* Add BPF to fuzz
* Implemented regs_read and regs_write
* Fix missing write-access on ALU's dst
* Updated cstool/, test_basic.c, test_detail.c, and test_iter.c
* Updated docs
* Fix type of cs_bpf#operands
* Implements python bindings
* Fix some bugs found by self code review
* Remove dummy tests
* remove typeof
* Address comments
* Fix MSVC's warnings and add test_bpf.py to bindings/python/Makefile
* Fix: call is not offset
* Added new M680X target. Supports M6800/1/2/3/9, HD6301
* M680X: Reformat for coding guide lines. Set alphabetical order in HACK.TXT
* M680X: Prepare for python binding. Move cs_m680x, m680x_insn to m680x_info. Chec
> k cpu type, no default.
* M680X: Add python bindings. Added python tests.
* M680X: Added cpu types to usage message.
* cstool: Avoid segfault for invalid <arch+mode>.
* Make test_m680x.c/test_m680x.py output comparable (diff params: -bu). Keep xprint.py untouched.
* M680X: Update CMake/make for m680x support. Update .gitignore.
* M680X: Reduce compiler warnings.
* M680X: Reduce compiler warnings.
* M680X: Reduce compiler warnings.
* M680X: Make test_m680x.c/test_m680x.py output comparable (diff params: -bu).
* M680X: Add ocaml bindings and tests.
* M680X: Add java bindings and tests.
* M680X: Added tests for all indexed addressing modes. C/Python/Ocaml
* M680X: Naming, use page1 for PAGE1 instructions (without prefix).
* M680X: Naming, use page1 for PAGE1 instructions (without prefix).
* M680X: Used M680X_FIRST_OP_IN_MNEM in tests C/python/java/ocaml.
* M680X: Added access property to cs_m680x_op.
* M680X: Added operand size.
* M680X: Remove compiler warnings.
* M680X: Added READ/WRITE access property per operator.
* M680X: Make reg_inherent_hdlr independent of CPU type.
* M680X: Add HD6309 support + bug fixes
* M680X: Remove errors and warning.
* M680X: Add Bcc/LBcc to group BRAREL (relative branch).
* M680X: Add group JUMP to BVS/BVC/LBVS/LBVC. Remove BRAREL from BRN/LBRN.
* M680X: Remove LBRN from group BRAREL.
* M680X: Refactored cpu_type initialization for better readability.
* M680X: Add two operands for insn having two reg. in mnemonic. e.g. ABX.
* M680X: Remove typo in cstool.c
* M680X: Some format improvements in changed_regs.
* M680X: Remove insn id string list from tests (C/python/java/ocaml).
* M680X: SEXW, set access of reg. D to WRITE.
* M680X: Sort changed_regs in increasing m680x_insn order.
* M680X: Add M68HC11 support + Reduced from two to one INDEXED operand.
* M680X: cstool, also write '(in mnemonic)' for second reg. operand.
* M680X: Add BRN/LBRN to group JUMP and BRAREL.
* M680X: For Bcc/LBcc/BRSET/BRCLR set reg. CC to read access.
* M680X: Correctly print negative immediate values with option CS_OPT_UNSIGNED.
* M680X: Rename some instruction handlers.
* M680X: Add M68HC05 support.
* M680X: Dont print prefix '<' for direct addr. mode.
* M680X: Add M68HC08 support + resorted tables + bug fixes.
* M680X: Add Freescale HCS08 support.
* M680X: Changed group names, avoid spaces.
* M680X: Refactoring, rename addessing mode handlers.
* M680X: indexed addr. mode, changed pre/post inc-/decrement representation.
* M680X: Rename some M6809/HD6309 specific functions.
* M680X: Add CPU12 (68HC12/HCS12) support.
* M680X: Correctly display illegal instruction as FCB .
* M680X: bugfix: BRA/BRN/BSR/LBRA/LBRN/LBSR does not read CC reg.
* M680X: bugfix: Correctly check for sufficient code size for M6809 indexed addressing.
* M680X: Better support for changing insn id within handler for addessing mode.
* M680X: Remove warnings.
* M680X: In set_changed_regs_read_write_counts use own access_mode.
* M680X: Split cpu specific tables into separate *.inc files.
* M680X: Remove warnings.
* M680X: Removed address_mode. Addressing mode is available in operand.type
* M680X: Bugfix: BSET/BCLR/BRSET/BRCLR correct read/modify CC reg.
* M680X: Remove register TMP1. It is first visible in CPU12X.
* M680X: Performance improvement + bug fixes.
* M680X: Performance improvement, make cpu_tables const static.
* M680X: Simplify operand decoding by using two handlers.
* M680X: Replace M680X_OP_INDEX by M680X_OP_CONSTANT + bugfix in java/python/ocaml bindings.
* M680X: Format with astyle.
* M680X: Update documentation.
* M680X: Corrected author for m680x specific files.
* M680X: Make max. number of architectures single source.
